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FORMS  OF  UNDERSTANDING  IN  MATHEMATICAL  PROBLEM  SOLVING 


James  G.  Greeno 
University  of  Pittsburgh 
Abstract 


Analyses  of  cognitive  structures  and  processes  involved  in 
performing  instructional  tasks  can  provide  objectives  for  instruction. 
This  chapter  focuses  on  performance  that  provides  evidence  of  students' 
understanding.  The  analyses  provide  hypotheses  about  knowledgethat 
constitutes  two  forms  of  understanding  in  the  domain  of, high  school 
geometry.  One  of  these  is  structural  understanding  in  Wertheimer's 
(1945/1959)  sense,  including  knowledge  that  enables  transfer  of 
knowledge  for  solving  problems  to  novel  kinds  of  problems.  The  other  is 
understanding  of  a  formal  principle  that  characterizes  an  important 
general  property  of  solutions  of  problems  in  a  domain. 
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FORMS  OF  UNDERSTANDING  IN  MATHEMATICAL  PROBLEM  SOLVING 


James  G.  Greeno 
University  of  Pittsburgh 


One  important  contribution  of  psychological  science  to  education 
has  been  to  provide  concepts  and  principles  for  formulating  behavioral 
objectives  in  much  of  the  school  curriculum.  As  we  achieve  greater 
understanding  of  cognitive  processes,  the  concepts  and  principles  that 
we  develop  can  be  used  in  formulating  objectives  of  instruction  in  more 
cognitive  terms.  These  objectives  are  in  the  form  of  cognitive  analyses 
of  instructional  tasks ;  Resnick  (in  press)  summarizes  several  examples 
of  such  analyses.  This  chapter  presents  two  examples  of  cognitive  task 
analysis  in  some  detail. 

The  theoretical  issue  studied  in  this  research  is  the  nature  of 
understanding.  The  goal  that  students  should  acquire  understanding  of 
concepts  and  principles  of  mathematics,  rather  than  merely  acquire  rote 
procedures,  is  widely  accepted.  However,  in  comparison  to  goals 
involving  concrete  skills  such  as  computational  algorithms,  objectives 
of  wider standing  have  not  been  formulated  in  definite,  specific  ways. 
Instructional  objectives  of  the  form,  "the  students  should  understand  [a 
concept]"  have  not  been  considered  sufficiently  specific  to  be  assessed 
behavlorally  and  therefore  have  played  a  decreasing  role  in 
instructional  design. 


In  Che  research  described  in  this  chapter,  the  goal  was  to  develop 
definite  theoretical  characterizations  of  understanding  in  a  specific 
domain  of  problems.  The  general  strategy  used  was  to  choose  some 
performance  chat  provides  persuasive  evidence  of  understanding  and  to 
develop  hypotheses  about  cognitive  structures  and  processes  that  cause 
the  performance  to  occur.  Assuming  that  the  performance  does  require 
understanding,  then  the  cognitive  structures  and  processes  that  bring  it 
about  constitute  understanding,  and  hypotheses  about  those  structures 
and  processes  are  hypotheses  about  the  nature  of  understanding.  If  the 
hypotheses  are  approximately  correct,  they  describe  knowledge  that 
constitutes  understanding,  and  thus  can  be  adopted  as  meaningful  and 
definite  cognitive  objectives  for  instruction. 

The  research  described  here  involved  analyses  of  performance  on  two 
tasks  in  the  domain  of  high  school  geometry.  These  two  analyses 
illustrate  several  general  points.  First,  the  analyses  provide  definite 
hypotheses  about  understanding,  illustrating  the  feasibility  of 
developing  such  hypotheses  for  school  tasks  and  showing  the  kinds  of 
cognitive  objectives  for  instruction  for  understanding  that  can  be 
obtained  from  such  analyses. 

A  second  point  is  one  emphasized  by  Resnick  (in  press).  She  notes 
that  cognitive  scientists  have  begun  to  give  more  attention  to  processes 
of  acquisition  and  instructional  intervention,  following  a  period  of 
almost  exclusive  focus  on  analyses  of  performance  of  relatively  complex 
tasks.  The  examples  I  discuss  here  are  consistent  with  that  trend,  in 
that  some  aspects  of  acquisition  and  instructional  intervention  are 
included  in  the  theoretical  analyses,  along  with  questions  about 


characteristics  of  cognitive  structures  and  processes  that  constitute 
understanding . 

A  third  point  is  that  analyses  of  knowledge  structures  Involved  in 
performing  school  tasks  can  address  questions  of  theoretical  interest  as 
well  as  potential  instructional  utility.  The  examples  I  discuss  are 
concerned  with  two  significant  theoretical  issues  in  the  psychology  of 
understanding . 

One  example  considers  knowledge  for  solving  problems  with 
structural  understanding.  Important  discussions  by  Judd  (1908),  by 
Katona  (1940),  and  especially  by  Wertheimer  (1945/1959)*  distinguished 
problem  solving  with  understanding  from  problem  solving  of  a  rote, 
mechanical  nature.  Previous  discussions  have  consisted  mainly  of 
examples  that  illustrate  the  phenomenon  of  structural  understanding  in 
compelling  ways.  A  goal  of  the  analysis  presented  here  is  to  clarify 
the  concept  of  structural  understanding  by  providing  a  definite  and 
specific  hypothesis  about  cognitive  structures  and  processes  that 
constitute  this  kind  of  understanding  in  the  context  of  a  specific 
problem  domain. 

The  second  example  considers  understanding  of  a  general  formal 
principle  related  to  solutions  of  specific  problems.  This  issue  has 

*Wertheimer's  analysis  is  the  most  widely  known;  indeed  it  would 
be  appropriate  to  use  the  label  "Wertheimer's  problem"  to  refer  to  the 
question  of  learning  to  solve  problems  with  understanding.  It  is  a 
pleasure  to  note  that  the  XXII  International  Congress  of  Psychology,  for 
which  the  Initial  version  of  this  Chapter  was  prepared,  was  held  very 
near  in  time  to  the  100th  anniversary  of  Wertheimer's  birth,  and  near 
his  birthplace  as  well.  Wertheimer  was  born  15  April  1880,  in  Prague. 


been  especially  salient  in  relation  to  questions  of  cognitive 
development,  where  Piaget's  (e.g.,  1941/1965)  research  was  interpreted 
as  indicating  that  young  children  lack  understanding  of  important 
general  logical  and  mathematical  concepts ,  but  more  recent  studies 
(e.g.,  Gelman  &  Gallistel,  1978)  indicate  that  significant  understanding 
of  those  concepts  should  be  attributed  to  young  children.  A  theoretical 
problem  arises  from  the  obvious  fact  that  this  understanding  cannot  be 
construed  as  explicit  knowledge  —  for  example,  preschoolers  obviously 
do  not  know  the  concept  of  cardinality  in  an  explicit  way.  The 
theoretical  problem  is  how  to  characterize  understanding  of  a  principle 
that  is  not  explicit.  The  analysis  presented  in  the  second  example 
provides  a  characterization  of  knowledge  that  constitutes  one  form  of 
implicit  understanding  of  a  formal  principle. 

A  fourth  general  point  illustrated  in  these  examples  is  that 
understanding  is  not  a  uniform  cognitive  state.  There  are  different 
knowledge  structures  that  constitute  understanding  in  different  forms. 
This  is  important  for  theory,  because  it  implies  that  there  will  not  be 
a  single  correct  cognitive  model  of  the  understanding  of  a  concept  or  a 
procedure.  It  also  is  important  for  educational  practice,  because  it 
Implies  that  goals  of  teaching  for  understanding  must  be  formulated  in 
more  precise  and  differentiated  terms  than  is  customary  in  order  to  play 
an  effective  role  in  instruction.  The  examples  presented,  involving 
structural  understanding  and  implicit  understanding  of  a  general 
principle,  involve  related  but  distinct  forms  of  understanding.  Both 
are  desirable  outcomes  of  instruction,  but  adoption  of  them  as 
instructional  objectives  would  lead  to  the  design  of  quite  different 
sets  of  materials  and  tasks. 


The  first  example  includes  theoretical  analyses  of  alternative 
knowledge  structures  that  can  be  acquired  in  learning  to  solve  some 
proof  exercises  that  are  included  in  geometry  courses.  One  alternative 
has  knowledge  that  constitutes  understanding  of  a  structure  of  relations 
in  the  problem;  the  other  lacks  that  understanding.  The  alternative 
that  simulates  knowledge  for  understanding  provides  a  definite 
hypothesis  in  which  problem-solving  procedures  are  integrated  with  a 
general  schema  for  part-whole  relationships .  The  analysis  also  provides 
a  definite  hypothesis  about  cognitive  structures  that  can  enable  such 
transfer  to  occur.  In  observations  reported  here,  substantial  variation 
was  found  among  students  in  their  performance  on  a  transfer  problem  that 
has  the  same  structure  as  a  set  of  problems  that  the  students  had 
previously  learned  to  solve.  In  the  model  that  was  developed  to 
represent  understanding  of  structure,  the  schematic  knowledge  that 
provides  the  basis  of  understanding  also  provides  a  basis  for 
transferring  problem-solving  procedures  to  novel  problems.  Thus,  the 
analysis  is  consistent  with  the  view  that  ability  to  transfer  knowledge 
to  novel  problems  provides  evidence  that  students  understand  the 
structure  that  the  problems  share. 

The  second  example,  involving  understanding  of  a  formal  principle, 
analyzes  implicit  understanding  of  the  principle  of  deductive 
consequence.  This  is  a  metaprinciple  in  relation  to  solutions  of  proof 
problems;  it  constitutes  a  general  criterion  of  validity  for  proofs. 
Understanding  of  the  principle  involves  knowledge  about  what  proofs  are, 
which  may  be  distinct  from  knowing  how  to  construct  correct  proofs  in 
specific  situations.  One  form  of  understanding  of  deductive  consequence 
is  knowledge  of  the  logical  requirements  for  a  deductively  valid 


inference.  Evidence  Chat  a  student  knows  these  requirements  can  be 
obtained  in  a  task  of  checking  proofs.  In  this  task,  the  student  must 
distinguish  valid  proofs  from  invalid  arguments;  thus  successful 
performance  requires  knowledge  of  the  defining  features  of  valid 
deduction.  In  the  study  reported  here,  we  first  found  that  students 
taking  a  high  school  geometry  course  did  not  acquire  this  understanding 
to  a  significant  degree.  Then  an  analysis  was  formulated  of  a  cognitive 
process  that  would  provide  a  basis  for  successful  performance  on  the 
task  of  proof  checking.  This  analysis  was  used  in  design  t,  instruction 
that  had  a  beneficial  effect  on  students'  performance  on  oof  checking, 
and  thus,  I  propose,  on  their  understanding  of  the  rinciple  of 
deductive  proof . 

Structural  Understanding 

The  analysis  of  structural  understanding  that  I  conducted  was 
focused  on  processes  of  learning  and  transfer.  Discussions  of 
structural  understanding  such  as  those  by  Judd  (1908),  Katona  (1940), 
and  Wertheimer  (1945/1959)  have  distinguished  meaningful  learning  from 
rote  learning  of  problem-solving  procedures.  Meaningful  learning  occurs 
when  the  material  that  is  learned  is  related  to  some  general  structure 
or  principle,  whereas  in  rote  learning  the  new  procedure  or  information 
is  simply  associated  with  the  specific  problem  situation  in  which  it  is 
experienced.  Evidence  for  understanding  resulting  from  meaningful 
learning  often  is  obtained  by  showing  that  after  meaningful  learning, 
individuals  are  better  able  to  transfer  their  knowledge  to  new  kinds  of 
problems . 


One  of  the  classical  examples  of  meaningful  learning  given  by 
Wertheimer  is  learning  the  proof  in  geometry  that  opposite  vertical 
angles  are  congruent.  Figure  1  shows  the  problem,  along  with 
representations  of  two  ways  to  think  about  the  solution.  Wertheimer 
pointed  out  that  children  often  learn  this  proof  in  a  mechanical  way, 
represented  by  the  equations  in  Figure  1,  memorizing  the  algebraic 
steps.  Wertheimer  contrasted  this  mechanical  kind  of  thinking  with  a 
more  meaningful  version,  in  which  the  proof  is  understood  in  relation  to 
spatial  relationships  among  the  angles .  He  used  a  diagram  like  the  one 
at  the  bottom  of  Figure  1  to  explain  these  relationships.  The  spatial 
pattern  can  be  seen  as  a  pair  of  overlapping  structures,  each  involving 
a  pair  of  angles  that  form  a  straight  line  and  containing  a  shared  part. 

Empirical  Observations 

I  present  some  data  that  illustrate  tl  ■».  range  of  understanding  that 
can  occur  regarding  problems  like  Figure  1.  The  data  consist  of 
thinking-aloud  protocols  that  were  obtained  from  six  students  during  the 
year  that  they  studied  geometry  in  high  school.  During  the  year,  each 
student  was  interviewed  approximately  once  a  week.  In  each  interview, 
the  student  solved  a  few  geometry  problems  and  thought  aloud  as  he  or 
she  worked  on  the  problems.  The  major  use  of  this  set  of  data  has  been 
in  developing  a  computational  model  that  simulates  the  problem-solving 
performance  of  the  students.  The  model  has  been  described  in  other 
reports  (Greeno,  1976,  1978;  Greeno,  Magone  &  Chaiklin,  1979). 


The  third  interview,  which  occurred  during  the  fifth  week  of  the 
course,  included  three  problems  Involving  the  structure  of  two  whole 
units  each  composed  of  two  p<-  :s ,  with  one  of  the  parts  shared  by  the 
two  wholes.  The  first  problem,  given  to  all  six  of  the  students,  is 
shown  in  Figure  2.  A  second  problem,  given  to  four  of  the  six  students 
after  they  worked  on  Figure  2,  is  about  segments,  rather  than  angles. 
It  is  shown,  along  with  its  solution,  in  Figure  3.  Finally,  the  problem 
of  vertical  angles,  shown  in  Figure  1,  was  given  to  four  of  the  six 
students . 

At  the  time  this  interview  occurred,  the  students  had  completed  a 
section  of  the  geometry  course  in  which  they  learned  to  solve  proof 
problems  involving  segments.  Thus,  problems  like  Figure  3  were  review 
problems  for  them;  in  fact  Figure  3  was  an  example  problem  in  the 
section  that  had  been  completed.  They  had  begun  to  learn  about  angles; 
concepts  such  as  right  angles,  adjacent  angles,  and  supplementary  and 
complementary  angles  had  been  introduced.  However,  proof  problems 
involving  angles  had  not  yet  been  studied  in  class.  One  of  the  students 
was  working  independently  and  had  done  some  proofs  involving  angles. 
For  the  other  five  students,  problems  such  as  Figures  1  and  2  were 
novel,  and  presented  tests  of  transfer  of  knowledge  about  solving 
related  but  distinct  problems,  such  as  Figure  3. 

To  anticipate  results  that  I  describe  later,  the  theoretical 
analysis  is  a  simulation  of  learning  that  can  occur  from  example 
problems  involving  segments.  Two  versions  of  learning  were  simulated: 
I  refer  to  these  as  stimulus-response  learning  and  meaningful  learning. 
The  difference  between  the  versions  is  a  hypothesis  about  the  knowledge 
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Statement 

1.  RONY 

2.  RN-OY 

3.  RN-RO  +  ON 

4.  OY-ON+NY 

5.  RO  +  ON-ON  +  NY 

6.  RO-NY 


Given:  RONY 
RN-OY 
Prove:  RO  -  NY 


Reason 

1.  Given 

2.  Given 

3.  Segment  addition  (Step  1 ) 

4.  Segment  addition  (Step  1 ) 

5.  Substitution  (Steps  2,  3,  4) 

6.  Subtraction  property  (Step  5) 


Figure  3.  A  problem  from  previous  learning  for  students,  and  the  third  example 
problem  for  simulation  of  learning. 


chat  enables  transfer  to  occur  between  problems  like  Figure  3  and 
problems  like  Figure  2. 

In  both  simulations  of  learning,  study  of  example  problems  leads  to 
acquisition  of  knowledge  used  In  solving  problems,  consisting  of 
procedures  for  writing  lines  of  proof.  The  simulations  differ  in  the 
representations  of  problems  that  provide  the  cues  for  use  of  the 
procedures.  In  the  simulation  of  stimulus-response  learning,  problems 
like  Figure  3  are  represented  quite  specifically;  the  cues  for  writing 
lines  of  proof  include  segments  that  are  perceived  in  the  diagram  of  a 
problem,  with  features  involving  the  ends  of  the  segments  that  are 
needed  for  the  additivity  of  their  lengths. 

In  the  simulation  of  meaningful  learning,  a  more  abstract 
representation  is  involved.  The  learner  represents  the  problem  using  a 
schema  that  identifies  part-whole  relations  among  the  objects  in  the 
problem.  The  problem-solving  procedures  that  are  acquired  have 
arguments  that  are  specified  in  terms  of  the  slots  of  the  part-whole 
schema.  This  enables  the  procedures  to  be  used  for  problems  that  have 
different  kinds  of  objects,  providing  that  the  objects  in  the  new 
problem  can  be  represented  using  the  schema  of  part-whole  relationships. 

For  example,  in  stimulus-response  learning,  a  procedure  is  acquired 
in  which  the  length  of  a  segment  is  subtracted  from  both  sides  of  an 
equation.  This  corresponds  to  Step  6  of  the  proof  in  Figure  3.  In 
meaningful  learning,  the  procedure  learned  from  Step  6  involves 
subtracting  a  part  that  is  shared  in  two  whole  units  from  each  of  them. 
The  conditions  for  applying  the  procedure  acquired  in  stimulus-response 
learning  include  the  existence  of  segments,  whereas  the  conditions  for 


applying  che  procedure  acquired  In  meaningful  learning  Include  objects 
designated  as  parts  and  wholes.  Thus,  the  procedure  acquired  In 
meaningful  learning  Is  potentially  applicable  in  a  new  problem  Involving 
angles,  whereas  the  procedure  acquired  In  stimulus-response  learning 
cannot  be  applied  unless  there  are  segments  in  the  problem. 

The  protocols  that  were  obtained  provided  an  impressive  range  of 
performance.  Some  of  the  students  gave  quite  clear  evidence  that  their 
representations  of  the  problems  included  the  part-whole  relationships 
that  provide  the  basis  for  transfer  from  segment  problems  like  Figure  3 
to  angle  problems  like  Figures  1  and  2.  Other  students  gave  quite  clear 
evidence  that  these  general  relationships  were  not  included  in  their 
representations . 

A  summary  of  the  six  students'  performances  on  Figure  2  is  in  Table 
1.  Two  students ,  S3  and  SS,  gave  proofs  that  seemed  to  result  directly 
from  use  of  the  overlap-part-whole  schema  described  previously.  These 
proofs  were  not  correct  technically;  they  involved  subtraction  of  the 
shared  angle  .dLBOC  before  statements  about  addition  of  angles  were 
included.  This  property  of  the  errors  is  consistent  with  the  hypothesis 
that  the  schema  was  used,  because  the  subtraction  procedure  is  a  salient 
component  of  the  schematic  knowledge.  For  a  correct  proof,  some 
lower-level  operations  have  to  be  performed  first,  and  it  is  plausible 
to  expect  that  the  schema  would  lead  students  to  think  of  the  top-level 
operations  earlier,  and  this  would  lead  to  the  kinds  of  errors  that  S3 
and  S5  committed.  Both  S3  and  S5  gave  correct  solutions  to  the  segment 
problem,  Figure  3,  and  remarked  about  the  similarity  of  that  problem  and 
Figure  2. 


Table  1 


Performance  on  Figure  2 


Performance 


Conceptually  correct  proofs  with  technical  er¬ 
rors.  Protocol  evidence  for  overlap-part-whole 
schema — e.g.,  "I  have  to  subtract."  "These  are 
the  same." 


Correct  proof.  No  protocol  evidence  for  schema 


Correct  proof  using  substitution  procedure.  Pr< 
tocol  evidence  against  schema:  "I'm  trying  to 
find  a  way  that  I  can  substitute. 


Failed  to  find  a  proof.  Protocol  evidence  for 
schema:  "I  would  have  to  subtract  .  .  .  would 
I  have  to  use  the  definition  of  betweenness?" 


No  progress  toward  proof. 


Two  students,  S2  and  S6,  gave  correct  proofs  but  did  not  provide 
evidence  of  using  the  schema.  S2  was  the  student  who  was  working 
independently  and  had  worked  on  proofs  about  angles.  There  was  no 
evidence  in  S2's  protocol  that  the  schema  was  used.  S6  worked  out  a 
proof  that  involved  an  analogy  to  proofs  for  segment  problems,  but  it 
apparently  was  based  on  the  substitution  procedure  rather  than  the 
structure  of  part-whole  relations.  S6  seemed  to  realize  that  if 
expressions  involving  addition  could  be  found,  they  could  serve  as 
arguments  in  a  procedure  that  involves  substitution,  and  solved  the 
problem  by  finding  those  expressions. 

A  fifth  student,  S4,  may  have  used  the  overlap-part-whole  schema 
for  representing  the  problem,  or  at  least  was  aware  of  the  general 
similarity  of  the  structure  of  Figure  2  and  problems  with  segments.  S4 
mentioned  subtraction  as  a  component  of  the  solution  and  asked  whether 
the  "definition  of  betweenness"  should  be  used.  "Definition  of 
betweenness"  was  the  name  given  in  the  course  text  for  addition  of 
lengths  of  segments,  and  S4's  use  of  this  rather  awkward  term  provided 
evidence  of  appreciating  the  similarity  between  Figure  2  and 
corresponding  problems  that  involved  segments.  S2  did  not  succeed  in 
solving  Figure  2.  S2  was  asked  to  solve  the  problem  with  segments  shown 
in  Figure  3  and  had  considerable  difficulty  with  it.  A  reasonable 
Interpretation  is  that  S2's  representation  of  Figure  2  may  have  included 
the  important  part-whole  relationships,  but  that  S2's  knowledge  of  the 
problem-solving  operations  was  too  weak  to  enable  a  solution  to  be  found 
for  the  problem. 


Performance  on  the  vertical  angles  problem,  Figure  1,  is  summarized 
in  Table  2.  This  problem  was  not  given  to  students  S4  and  SI. 

When  the  vertical  angles  problem  was  presented,  S3  gave  strong 
evidence  of  understanding  its  relation  to  Figure  2  and  Figure  3,  saying 
"You  know  something,  I'm  getting  sort  of  tired  of  that  problem."  S3  gave 
a  proof  of  the  vertical  angles  theorem  that  was  similar  to  the  one  that 
S3  gave  for  Figure  2,  with  subtraction  incorrectly  used  before  addition 
of  angles  was  asserted. 

S5  did  not  succeed  in  finding  a  proof  for  the  vertical  angles 
theorem.  However,  there  was  further  evidence  that  S5  had  used  the 
overlap-part-whole  relationships  in  solving  Figure  2.  When  that  problem 
was  shown  again  and  S5  was  asked  whether  a  similar  method  could  be  used 
for  Figure  1,  S5  apparently  saw  that  the  structure  provided  a  way  to 
proceed,  saying,  "I  could  say  if  I  had  this,  I  could  subtract  the 
supplementary  angle  from  that  one." 

S2  gave  a  correct  proof  for  the  vertical  angles  theorem,  using  a 
theorem  that  two  angles  that  are  supplementary  to  the  same  third  angle 
are  congruent  to  each  other.  When  S2  was  asked  whether  there  was  a 
connection  between  the  vertical  angles  theorem  and  Figure  2,  S2's 
response  did  not  provide  evidence  for  use  of  the  overlap-part-whole 
schema.  S2  assimilated  Figure  2  to  the  solution  scheme  used  for  the 
vertical  angles  theorem,  saying  that  Figure  2  could  be  solved  using 
complementary  angles. 


Table  2 


Performance  on  Figure  1 


Performance 

Proof  with  technical  errors  like  Figure  2.  Pro¬ 
tocol:  "It's  the  same  problem  again;  I'm  getting 
sort  of  tired  of  that  problem." 


Did  not  find  a  proof.  Saw  the  way  to  proceed 
using  subtraction  when  Figure  2  was  shown  again. 


Correct  proof  using  supplementary  angles.  Found 
an  analogous  proof  involving  complementary  angles 
for  Figure  2. 


Did  not  find  a  proof.  Was  trapped  by  perceptual 
distraction. 


S6  was  unable  to  prove  the  vertical  angles  theorem,  and  gave 
performance  that  Wertheimer  (1945/1959)  noted  as  evidence  for  a  lack  of 
structural  understanding.  S6  proceeded  with  the  problem  by  noting  that 
X  +  Y  ■  180°  and  that  W  +  Z  ■  180* .  There  probably  are  perceptual 
factors  that  produce  the  tendency  to  think  of  the  problem  in  this  way. 
S6  was  trapped  in  this  view  of  the  problem  and  never  escaped  from  it.  A 
representation  with  two  distinct  pairs  of  angles  is  inconsistent  with 
the  overlap-part-whole  schema,  so  S6's  performance  provides  further 
strong  evidence  that  those  relationships  were  not  in  S6's 
representation . 

In  summary,  three  students  gave  quite  strong  evidence  that  the 
structure  of  relationships  in  the  overlap-part-whole  schema  was  in  their 
representation  of  Figure  2.  None  of  these  students  gave  proofs  that 
were  entirely  correct;  however,  S3  and  S5  gave  proofs  that  showed  a 
good  grasp  of  the  problem.  S3  showed  good  transfer  to  the  vertical 
angles  problem,  and  S5  recognized  the  structure  of  that  problem  when 
Figure  2  was  shown  again.  The  third  of  these  students,  S4,  failed  to 
find  a  proof  for  Figure  2,  apparently  because  of  weak  knowledge  of 
problem-solving  procedures.  A  fourth  student,  S2,  solved  both  Figure  2 
and  the  vertical  angles  problem  successfully,  but  did  not  provide  any 
protocol  evidence  of  using  the  overlap-part-whole  schema.  It  is 
possible,  of  course,  that  the  relationships  in  that  structure  were 
recognized  and  used  by  S2  in  an  implicit  way.  A  fifth  student,  S4, 
solved  Figure  2  successfully  and  gave  relatively  clear  evidence  of  not 
being  cognizant  of  the  overlap-part-whole  relationships,  especially  in 
the  vertical  angles  problem  that  S4  was  unable  to  solve.  The  sixth 
student,  SI,  made  no  progress  on  any  of  the  problems,  apparently  lacking 


a  great  deal  of  the  knowledge  required  for  successful  performance. 

Theoretical  Analysis 

The  goal  of  the  theoretical  analysis  was  to  Identify  a  set  of 
learning  processes  and  knowledge  structures  that  could  simulate 
acquisition  of  learning  with  structural  understanding.  In  developing 
this  model  1  collaborated  with  John  Anderson;  some  of  the  results  were 
presented  in  Anderson,  Greeno,  Kline,  &  Neves  (1981)  as  part  of  a 
general  discussion  of  the  acquisition  of  problem-solving  skill. 

To  clarify  the  specific  structures  responsible  for  understanding, 
two  versions  of  the  learning  simulation  were  formulated.  One  of  these 
is  called  stimulus-response  learning,  and  the  new  problem-solving 
procedures  acquired  by  this  system  are  associated  with  relatively 
specific  situational  cues.  In  the  other  system,  called  meaningful 
learning,  the  new  problem-solving  procedures  are  acquired  as  integral 
parts  of  schemata  that  represent  problem-solving  situations  in  terms  of 
general  part-whole  relations. 

The  models  that  were  formulated  simulate  learning  of  procedures 
from  worked-out  examples,  as  do  many  recent  computational  models  of 
learning  procedures  (Neves,  1981;  Vere,  1978).  The  learning  that  was 
simulated  is  based  on  two  tasks  that  are  given  early  in  the  geometry 
course  and  one  task  that  produces  learning  that  should  occur  some  years 
earlier . 
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The  first  two  example  problems  used  for  simulation  of  learning  are 
shown  in  Figure  4,  and  the  third  example  is  shown  in  Figure  3.  The 
examples  were  used  to  investigate  three  different  aspects  of  learning  in 
which  new  material  is  related  to  a  schema.  In  the  first  problem, 
meaningful  learning  involves  an  existing  schema  and  the  model  simulates 
learning  to  apply  that  schema  in  a  situation  where  it  was  not  applicable 
previously.  Meaningful  learning  in  the  second  problem  involves 
elaborating  an  existing  schema  by  forming  new  problem-solving  procedures 
that  become  part  of  the  schema.  In  the  third  problem  situation, 
meaningful  learning  involves  building  a  new  schema,  which  has  previously 
existing  schemata  as  subschemata.  In  all  three  cases,  rote  learning  was 
simulated  by  a  process  that  acquires  problem-solving  procedures  and 
associates  them  with  relatively  specific  representations  of  the  problem 
situation. 

The  simulation  models  were  programmed  in  ACTP  (Greeno,  1978),  a 
variant  of  Anderson's  (1976)  ACT  production  system.  In  this  system, 
there  is  declarative  knowledge  represented  by  a  semantic  network  and 
procedural  knowledge  in  the  form  of  production  rules.  In  learning  from 
examples,  the  input  for  learning  is  the  solution  of  a  problem,  and  the 
system  learns  by  adding  productions  to  its  procedural  knowledge  or  by 
adding  semantic-network  structures  to  the  declarative  knowledge  that  it 
has  available  for  solving  problems.  The  procedures  for  learning  used  in 
this  simulation  were  not  the  same  as  those  developed  and  investigated  by 
Anderson  in  his  use  of  ACT  as  a  model  of  learning  (Anderson,  1981). 


Given:  AB  =  5 
BC  *  3 
Find:  AC 


Solution:  AC  -  5  +  3  -  8 


Statement 

1  1.  ABC 

2.  AB  +  BC  -  AC 

3.  AB  =  AC  —  BC 

I 


Reason 

1.  Given 

2.  Segment  addition  (Step  1) 

3.  Subtraction  property  (Step  2) 


Figure  4.  First  two  example  problems  for  simulation  of  learning. 
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For  learning  new  productions,  a  process  was  developed  for  keeping  a 
representation  of  the  problem  situation  in  working  memory.  This 
representation  determines  the  information  that  is  included  in  the 
conditions  of  new  productions  when  they  are  added  to  the  system.  The 
representation  includes  semantic  information  that  is  taken  from  the 
problem  situation  (either  the  diagram  or  the  system's  semantic 
knowledge)  that  increases  the  coherence  of  the  problem  representation. 
The  process  for  selecting  the  added  semantic  information  is  similar  to 
that  used  by  Kintsch  and  VanDijk  (1978)  for  forming  coherent 
representations  in  their  model  of  text  comprehension,  although  the  one 
developed  for  ACTP  adds  more  information  than  Kintsch  and  VanDijk' s 
does . 


Processes  also  were  implemented  for  schema-based  learning.  The 
process  for  learning  to  apply  an  existing  schema  in  a  new  situation  uses 
the  problem  representation  in  working  memory  to  form  the  condition  of 
the  schematizing  production,  and  uses  the  action  performed  in  the 
example  to  determine  how  to  assign  components  of  the  situation  to  slots 
in  the  schema.  Acquisition  of  new  procedures  and  synthesis  of  a  new 
schema  required  knowledge  of  general  structural  features  of  schemata  in 
the  system.  This  knowledge  constitutes  a  kind  of  metaschema — a  schema 
that  enables  the  system  to  acquire  new  schematic  knowledge.  For 
example,  procedures  attached  to  schemata  include  information  about 
consequences  that  are  matched  against  goals  to  determine  whether  use  of 
the  procedure  is  desirable,  and  information  about  prerequisites  that  are 
tested  in  the  problem  situation  to  determine  whether  the  procedure  can 
be  applied.  The  process  for  acquiring  new  procedures  identifies 
conditions  in  the  problem  situation  that  are  included  as  prerequisites. 
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and  a  generalized  form  of  Che  action  in  Che  example  is  included  as  a 
consequence  of  Che  procedure.  The  process  for  synthesizing  a  new  schema 
also  includes  knowledge  about  Che  general  structure  of  schemata,  in  the 
form  of  actions  that  store  new  semantic  knowledge  in  the  form  required 
by  the  procedures  that  retrieve  and  utilize  schemata  for  problem 
solving. 

The  schemata  that  are  critical  for  the  system's  meaningful  learning 
are  included  in  its  declarative  knowledge.*  The  structure  of  schematic 
knowledge  in  this  model  is  not  unusual;  it  is  a  somewhat  simplified 
version  of  the  concept  that  Bobrow  and  Winograd  (1977)  used  in  the 
Knowledge  Representation  Language  (KRL).  The  main  components  are  listed 
in  Table  3.  Like  all  schemata,  those  used  here  provide  a  set  of 
relationships  among  some  objects.  The  objects  fit  into  the  structure  in 
positions  called  slots.  There  are  procedures  for  applying  the  schema  in 
specific  situations;  these  are  referred  to  as  schematizing  productions. 
There  is  some  information  that  identifies  features  of  the  situation  that 
are  relevant  for  performing  procedures;  these  are  called  contextual 
associations.  An  important  feature  of  these  schemata  is  that  they  have 
procedures  associated  with  them,  in  the  manner  of  KRL.  The  organization 
of  these  procedure  descriptions  is  patterned  after  Sacerdoti's  (1977) 
model  of  planning  in  problem  solving,  called  Network  of  Action 


*1  consider  the  use  of  declarative  structure  for  representing 
schemata  as  a  detail  of  implementation,  rather  than  a  substantive 
psychological  hypothesis.  In  fact,  I  am  inclined  to  believe  that  the 
relationships  incorporated  in  these  schemata  probably  are  represented  in 
humans  as  cognitive  procedures,  rather  than  as  declarative  structures. 
I  believe  that  a  model  could  be  formulated  in  which  schemata  would  be 
represented  procedurally,  and  that  it  would  be  functionally  equivalent 
to  the  model  that  I  describe  here. 
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Table  3 

General  Components  of  Schemata 


Schema 

Slots 

Schematizing  Productions 
Contextual  Associations 
Procedural  Attachments 
—  Prerequisites 
—  Performances 


—  Consequences 


Hierachies  (NOAH) .  Information  stored  about  each  procedure  includes 
prerequisite  conditions,  consequences,  and  specific  actions  that  are 
carried  out  in  performing  the  procedure. 

Learning  New  Applicability  Conditions.  The  first  example  problem 
that  the  model  learns  from  is  the  first  problem  in  Figure  3.  I  assume 
that  students  learn  to  solve  problems  like  this  some  years  before  they 
study  geometry.  The  problem  was  included  in  the  analysis  to  clarify  the 
requirement  of  having  knowledge  about  applying  schemata  to  represent 
problems . 

In  the  simulation  of  stimulus-response  learning  from  the  first 
example  problem,  the  action  of  adding  lengths  is  associated  with  a 
representation  of  the  problem  situation  that  specifies  some  relevant 
properties  such  as  the  collinearlty  of  the  segments.  The  knowledge 
acquired  by  rote  learning  enables  the  system  to  solve  new  problems  that 
are  closely  similar  to  the  one  in  which  learning  occurred,  but 
generalization  is  very  limited.  For  example,  the  rote  learner  does  not 
generalize  to  a  problem  in  which  the  total  length  and  one  subsegment  are 
given,  and  the  other  subsegment  is  to  be  calculated. 

Meaningful  learning  occurs  in  the  first  example  problem  when  there 
is  an  active  schema  that  is  assumed  as  prior  knowledge:  the 
relationship  between  parts  that  are  combined  to  form  a  whole.  The 
components  of  this  schema  are  shown  in  Table  4.  We  have  evidence  from 
studies  of  children's  performance  on  arithmetic  story  problems  (Riley, 
Greeno,  &  Heller, in  press)  that  this  schema  has  been  acquired  by  most 
8-year-olds.  The  schema  is  used  in  understanding  problems  such  as  the 
following:  "There  are  five  girls  and  eight  children;  how  many  boys  are 


Table  4 

Components  of  Whole/Parts  Schema 


Whole/Parts 

Slots:  Whole,  Part  1,  Part  2 
Context:  Set— ——►Number 
Schematize  ion: 

Set  1 

Set  3 

Procedures: 

Combine/Calculate 
Separate/Calculate 


Adjust  Parts 


there?"  The  schematizing  production  identifies  the  set  of  boys  and  the 
set  of  girls  as  the  parts  and  the  set  of  children  as  the  whole.  Then, 
using  the  information  stored  in  procedural  attachments,  the  procedure  of 
separation  is  chosen  and  the  numbers  are  subtracted  to  find  the  answer. 

When  the  system  learns  meaningfully  from  the  first  example  problem 
it  learns  to  apply  this  schema  to  problems  involving  segments.  It 
acquires  a  new  schematizing  production,  which  is  sketched  in  Figure  5. 
As  a  result,  in  future  problems  like  this  one  the  model  can  interpret 
the  segments  as  a  part-whole  structure.  One  consequence  is  that  the 
whole  collection  of  procedures  associated  with  the  schema  is  then 
available  for  solving  a  variety  of  problems,  which  is  not  the  case  when 
simple  stimulus-response  learning  occurs.  For  example,  the  knowledge 
acquired  in  meaningful  learning  generalizes  to  problems  in  which  the 
total  length  and  one  subsegment  are  given  and  the  other  subsegment  is  to 
be  calculated,  because  when  the  whole/parts  schema  is  applied,  the 
procedure  of  separating  the  whole  into  parts  is  made  available. 

Learning  New  Procedural  Attachments.  The  second  example  is  the 
second  problem  in  Figure  4.  From  this  problem  the  model  learns 
procedures  required  for  writing  steps  of  proof.  (Previously,  the 
model's  only  procedures  involved  calculations  with  numbers.)  The 
stimulus-response  learning  system  simply  learns  actions  of  writing  lines 
of  proof  like  Steps  2  and  3  in  the  example  when  the  stimulus  conditions 
are  equivalent  to  those  of  the  example  problem,  including  collinear 
points  and  a  goal  to  prove  that  the  length  of  one  segment  is  the 
difference  between  lengths  of  two  other  segments. 
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Learned  schematizing  production: 


Condition 


Endpoint  1 


Segment  3 


Action 


Apply  Whole/Parts  schema,  with 

Segment  1  *  Part  1 
Segment  2  ■  Part  2 
Segment  3  ■  Whole 


Figure  5.  Schematizing  production  learned  from  the  first  example  with  meaningful 


Meaningful  learning  from  Che  second  example  involves  an  addition  of 
structure  to  the  part-whole  schema  that  is  already  known  by  the  system. 
The  new  structures  are  procedures  that  are  associated  with  the  schema. 
They  are  sketched  in  Table  5.  Note  that  the  prerequisites  of  these 
procedures  include  problems  that  have  been  schematized  as  part-whole 
structures.  The  actions  of  the  procedures  are  defined  in  terms  of  the 
slots  of  the  whole/parts  schema.  This  means  that  the  actions  involve 
writing  steps  of  proof  that  refer  to  objects  that  occupy  the  slots  of  an 
Instantiation  of  the  schema.  The  main  consequence  of  this  learning,  in 
contrast  to  the  stimulus-response  learning  from  this  example,  is  that 
the  procedures  that  are  learned  are  applicable  in  any  problem  situation 
that  is  represented  using  the  whole/parts  schema.  The  procedures 
acquired  in  stimulus-response  learning  can  only  be  applied  in  situations 
involving  segments  with  collinear  endpoints. 

Learning  a_  New  Schema .  The  final  learning  task  that  was  simulated 
used  the  problem  in  Figure  3.  Stimulus-response  learning  from  this 
example  acquires  the  procedures  for  writing  proof  steps  involving 
substitution  and  the  subtraction  property,  based  on  Steps  3  and  6,  and 
associating  them  with  relatively  specific  problem  conditions,  as  in  the 
earlier  examples. 

The  meaningful  version  of  learning  from  this  example  results  in  a 
new  schema,  sketched  in  Table  6,  in  which  two  part-whole  schemata  are 
included  as  subschemata.  The  problem  solver  represents  the  segments  RO, 
ON,  and  RN  as  a  part-whole  structure,  and  also  represents  the  segments 
ON,  NY,  and  OY  as  a  second  part-whole  structure.  This  enables  it  to 
apply  the  procedures  for  writing  steps  based  on  segment  addition, 


Table  5 

Procedures  Learned  from  the  Second  Example 


Table  6 

Schema  Acquired  from  Figure  3 


Overlay/Who  le/Parts 
Slots: 

Part  1,  Part  2,  Part  3,  Whole  1,  Whole  2 
Subschemata: 

Whole/Parts  1,  (Parts  1,  Part  2,  Whole  1 ) 
Whole/Parts  2  (Part  2,  Part  3,  Whole  2) 

Procedures: 

Write  ■  Substitution 
Write  -  Subtraction  (2  sides) 


corresponding  to  Steps  3  and  4  of  the  example.  Steps  5  and  6  of  the 
example  Involve  new  procedures  for  the  model.  They  also  involve  both  of 
the  part-whole  structures  represented  in  the  problem.  This  leads  to  the 
formation  of  a  new  schema,  with  two  part-whole  structures  that  share  one 
of  their  parts.  Problem-solving  procedures  corresponding  to  Steps  5  and 
6  of  the  example  are  acquired  and  associated  with  the  new  schema. 

Conclusions 

The  simulation  of  meaningful  learning  provides  a  definite 
hypothesis  about  a  process  that  can  acquire  problem-solving  procedures 
with  structural  understanding.  Although  transfer  to  problems  about 
angles  was  not  implemented,  it  is  clear  that  the  knowledge  acquired  in 
meaningful  learning  from  examples  about  segments  is  applicable  in 
problems  like  Figures  1  and  2.  For  a  student  to  make  that  application, 
schematizing  productions  would  have  to  be  learned  to  enable  the 
appropriate  schemata  to  be  used  to  represent  relationships  among  angles, 
or  the  student  could  use  more  general  interpretive  processes  to 
comprehend  the  problems  as  instances  of  the  overlap-part-whole 
structure . 

It  is  significant  that  a  plausible  simulation  of  meaningful 
learning  could  be  formulated  using  only  knowledge  structures  of 
relatively  standard  kinds.  Schemata  with  characteristics  that  are 
standard  in  knowledge  representation  systems  and  learning  processes  that 
are  commonly  proposed  for  learning  from  examples  were  combined  in  a 
straightforward  way  to  form  a  simulation  of  learning  with  significant 
structural  understanding.  This  seems  quite  encouraging  for  the  prospect 


of  developing  definite  hypotheses  about  understanding  over  a  substantial 
range  of  instructional  tasks. 


Understanding  a  General  Formal  Principle 

The  second  analysis  I  present  was  concerned  with  a  somewhat 
different  aspect  of  understanding,  involving  a  formal  principle.  This 
study,  in  which  I  collaborated  with  Maria  Magone,  was  concerned  with 
geometry  students'  understanding  of  the  general  concept  of  proof.  This 
constitutes  a  metaprinciple  in  relation  to  the  tasks  that  students  are 
taught  to  perform  in  their  study  of  geometry.  The  skills  that  students 
are  required  to  learn  involve  constructing  proofs.  The  question 
addressed  in  this  study  is  whether  students  understand  what  proofs 
are — that  is,  whether  they  know  the  general  features  that  are  required 
for  something  to  be  a  proof. 

The  concept  we  focused  on  can  be  called  the  principle  of  deductive 
consequence.  In  a  formal  proof,  each  assertion  that  is  made  is 
deductively  required  by  the  premises  of  the  problem  or  by  other 
assertions  that  have  been  made  explicitly.  This  imposes  a  strong 
criterion  for  statements  to  be  acceptable  for  inclusion  in  a  proof.  For 
each  statement  that  is  put  into  a  proof,  there  must  be  a  sufficient 
basis  in  earlier  statements  that  the  added  statement  must  be  true  if  the 
earlier  statements  are  accepted.  This  criterion  contrasts  with  ordinary 
exposition,  in  which  statements  are  expected  to  be  reasonable  in  the 
light  of  previous  information — for  example,  new  statements  should  not 
directly  contradict  statements  made  earlier — but  there  is  not  a  general 
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constraint  that  each  new  statement  is  required  by  the  assertions  made 
previously. 

The  task  that  we  used  to  investigate  understanding  of  deductive 
consequence  involved  checking  proofs.  We  presented  proof  problems  with 
solutions  worked  out,  and  asked  students  to  check  whether  each  proof  was 
correct.  If  the  student  said  that  there  was  an  error,  we  asked  what  was 
wrong . 

Our  study  had  three  parts.  First,  we  gave  proof-checking  problems 
to  some  students  who  were  taking  a  course  in  geometry.  (These  were 
different  students  than  those  who  gave  the  protocols  described  in  the 
first  section.)  Six  problems  were  given  in  interviews  held  in  November, 
and  four  of  the  same  problems  were  given  in  May.  Fifteen  students 
participated  each  time,  with  ten  students  participating  in  both 
interviews  and  five  additional  students  who  were  different  in  the  two 
interviews.  The  performance  of  students  on  proof-checking  tasks  was  not 
very  good,  especially  on  a  problem  in  which  the  error  to  be  detected  was 
that  an  assertion  needed  to  justify  a  step  had  been  omitted. 

The  second  part  of  our  study  was  a  theoretical  investigation r  We 
developed  a  computational  model  of  correct  performance  on  the 
proof -checking  task,  including  problems  with  errors  of  omitted 
information.  Because  the  students  whom  we  observed  had  not  performed 
these  tasks  successfully,  the  model  we  developed  was  not  a  simulation  of 
performance  which  had  been  observed.  ever,  we  attempted  to  develop  a 
model  that  was  psychologically  plausible,  in  the  sense  that  it  used 
procedural  and  propositional  knowledge  that  we  felt  could  be  acquired  by 
human  learners,  and  would  simulate  performance  of  human  problem  solvers 


who  had  acquired  chat  knowledge 


The  third  part  of  the  9tudy  was  an  instructional  experiment  to  test 
whether  the  proof -checking  knowledge  incorporated  in  the  model  could  be 
acquired  by  human  learners.  We  developed  instructional  materials  for 
training  the  procedure  of  checking  proofs.  A  group  of  15 
college-student  subjects  who  had  studied  geometry  in  high  school 
received  this  instruction,  along  with  sufficient  review  of  basic 
geometry  to  make  the  instruction  possible.  A  control  gi  up  was  given 
only  the  review  materials.  The  experimental  group  succeeded  better  on 
proof  checking  and  other  geometry  problems  than  the  control  group,  both 
in  the  domain  of  problems  that  was  used  in  the  instruction  and  in 
another  domain  of  problems  that  was  used  in  a  test  of  transfer. 

Performance  on  Proof  Checking 

One  of  the  proof-checking  problems  that  we  used  is  shown  in  Figure 
6.  The  proof  shown  is  not  valid.  For  Step  1  to  be  justified,  lines  AC 
and  BD  should  have  been  asserted  as  parallel.  Because  this  was  neither 
given  nor  proved  in  prior  steps.  Step  1  is  not  justified. 

Performance  of  the  geometry  students  on  proof-checking  problems  is 
shown  in  Table  7 .  The  problem  in  Figure  6  was  the  one  that  is  called 
"Missing  given"  in  Table  7.  Note  that  performance  on  that  problem  was 
poor  both  in  November  and  in  May.  Performance  on  other  problems  was 
somewhat  better.  The  valid  proof  had  a  diagram  in  which  the  statement 
to  be  proved  appeared  false;  apparently  some  students  came  to  disregard 
the  appearance  of  the  diagram  between  November  and  May.  Other  sources 
of  errors  included  reasons  given  in  the  problem  that  are  not  theorems  of 
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Given:  aE  /  CE  and  AB  a  CD 
Prove:  L  ACD  a  L  ABD 


Statement 

Reason 

1. 

L  ACD  a  L  BDE 

corresponding  Ls 

2. 

<LBDE  a  L  ABD 

alternate  interior's 

3. 

L  ACD  a  L  ABD 

transitive  property 

Figure  6.  A  difficult  proof-checking  problem. 


geometry,  and  reasons  whose  antecedents  did  not  apply  to  the  objects 
that  the  statements  referred  to.  In  two  problems,  the  diagrams  were 
consistent  with  the  statements  to  be  proved;  in  the  other  two  problems, 
the  statements  to  be  proved  were  incorrect  in  the  diagrams.  About  half 
the  students  found  the  errors  in  the  problems  with  consistent  diagrams, 
both  in  November  and  in  May.  Performance  in  November  was  somewhat 
better  on  the  problems  where  the  discrepancy  could  be  seen  visually; 
these  problems  were  not  included  in  the  May  interviews. 

Theoretical  Analysis 

The  goal  of  the  theoretical  analysis  was  to  develop  a  definite 
hypothesis  about  knowledge  that  would  enable  students  to  solve 
proof -checking  problems  successfully.  To  do  this  we  formulated  a 
computational  model  of  correct  proof-checking  performance. 

The  procedure  that  we  implemented  is  sketched  in  Table  8.  This 
procedure  is  applied  to  each  step  in  the  proof.  If  all  the  steps  are 
accepted,  and  the  final  step  corresponds  to  the  problem  goal,  then  the 
proof  is  accepted. 

First,  the  reason  that  is  given  in  the  proof  step  is  checked 
against  a  list  of  theorems  that  are  available  for  use.  (The  list 
includes  definitions  and  postulates  as  well  as  theorems;  the  model  does 
not  recognize  this  technical  distinctive. )  Each  theorem  is  associated 
with  its  "deep  structure,"  in  which  the  antecedent  and  consequent  of  the 
conditional  proposition  are  represented  with  variables  that  can  be 
matched  with  objects  in  the  statement  and  in  previous  statements  and  the 
given  information. 
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Table  8 

Procedure  for  Verifying  a  Step 


(a)  Test  whether  reason  is  a  theorem 
lb)  Retrieve  deep  structure  of  reason 

(c)  Instantiate  consequent  in  statement 

(d)  Form  corresponding  instantiation  in  antecedent 

(e)  Determine  whether  antecedent  has  been  asserted 

(f)  Determine  whether  statement  matches  problem  goal 


The  second  seep  of  Che  procedure  involves  retrieving  Che  antecedent 
and  consequent  of  the  theorem  that  is  given  as  the  reason.  Then  there 
is  a  matching  process,  in  which  he  consequent  of  the  theorem  is  matched 
with  the  statement  and  the  variables  in  the  consequent  are  replaced  by 
the  objects  that  are  mentioned  in  the  statement.  If  the  consequent 
cannot  be  matched  to  the  statement,  the  step  is  rejected.  This  happens 
if  the  property  or  relation  asserted  in  the  statement  does  not  match  the 
property  or  relation  of  the  consequent. 

In  the  fourth  step  of  the  procedure,  the  objects  that  were  found  in 
the  statement  are  substituted  for  the  variables  in  the  antecedent, 
forming  one  or  more  propositions  that  should  have  been  asserted  in 
previous  steps  or  in  the  given  information.  Finally,  there  is  a  search 
for  those  propositions  in  the  set  of  previous  statements  and  given 
information  of  the  problem.  If  they  are  found,  the  proof  step  is 
accepted;  otherwise,  it  is  rejected. 

To  clarify  the  procedure,  consider  its  application  to  the  first 
step  of  the  problem  in  Figure  6.  First,  the  reason  "Corresponding 
angles"  is  checked  with  the  list  of  available  theorems.  This  is 
successful;  there  is  a  theorem  (actually,  a  postulate)  called 
"Corresponding  angles"  on  the  list. 

Next,  the  deep  structure  of  this  theorem  is  retrieved.  The 
antecedent  is  "If  angles  X  and  Y  are  corresponding  angles  with  lines  L 
and  M  as  sides,  and  lines  and  M  are  parallel,  and  line  tl  contains 
their  other  sides,"  and  the  consequent  is,  "Then  X  and  Y  are  congruent." 
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In  Che  third  step,  the  consequent  of  the  reason  is  matched  with  the 
statement  of  the  proof  step.  The  step  says  that  ^.ACD  and  ^.BDE  are 
congruent.  Therefore,  X  and  If  in  the  consequent  are  replaced  by  these 
angles . 

In  the  fourth  step,  the  propositions  that  have  to  be  asserted  are 
formed.  This  includes  a  pattern  in  which  ^.ACD  and  Z.BDE  are 
corresponding  angles,  with  two  of  their  respective  sides  collinear. 
There  also  must  be  an  assertion,  either  in  given  information  or  a 
previous  statement,  that  the  other  sides  are  parallel. 

Finally,  in  a  search  for  these  propositions,  the  system  fails  to 
find  the  assertion  about  parallel  lines  that  is  needed.  The  system 
rejects  the  first  step  of  the  proof. 

The  knowledge  required  for  checking  proofs  can  be  compared  with 
knowledge  for  constructing  proofs,  especially  as  a  model  for  solving 
proof  construction  problems  was  formulated  earlier  (Greeno,  1978).  The 
knowledge  needed  for  proof  checking  is  much  simpler  in  one  way  than  the 
knowledge  that  is  needed  for  constructing  proofs,  but  is  somewhat  more 
complex  in  two  other  ways . 

The  factor  that  makes  knowledge  for  checking  proofs  simpler  is  that 
steps  of  the  proof  are  already  shown.  This  eliminates  the  need  to 
search  for  a  sequence  of  problem-solving  inferences  to  achieve  the  goal 
of  the  problem.  As  a  result,  the  strategic  knowledge  used  for  choosing 
plans  and  goals  that  guide  the  search  for  inferential  operators  is  not 
needed  for  proof  checking. 


28 


T 


On  the  other  hand,  two  components  of  knowledge  are  required  for 
proof  checking  that  are  not  needed  for  constructing  proofs.  First,  the 
procedures  for  verifying  steps  of  proof  and  checking  whether  the  main 
goal  has  been  satisfied  must  be  added.  The  other  complexity  Involves 
the  way  In  which  postulates  and  theorems  are  represented  In  the  models 
of  proof  construction  and  checking.  In  the  model  of  proof  construction, 
postulates  and  theorems  are  represented  Implicitly  as  problem-solving 
operations  In  the  form  of  production  rules,  with  the  antecedents  as 
conditions  that  are  tested  and  the  consequents  as  actions  that  assert 
properties  or  relations  that  are  Inferred.  However,  in  the  model  of 
proof  checking  there  Is  an  explicit  representation  of  postulates  and 
theorems,  which  are  stored  In  a  list  so  that  reasons  can  be  checked  to 
determine  whether  they  are  theorems  and  their  antecedents  and 
consequents  can  be  retrieved  In  the  process  of  determining  whether  the 
needed  assertions  have  been  made. 

Instructional  Experiment 

We  performed  a  test  of  our  ideas  about  proof  checking  by  designing 
some  Instructional  materials  based  on  the  computational  model  of  proof 
checking  that  we  had  implemented.  This  Instruction  was  given  to  some 
human  learners.  The  subjects  were  able  to  use  the  procedure  that  was 
taught  in  the  instruction,  and  it  resulted  in  some  improvement  in  their 
performance  in  proof  checking.  We  take  this  as  evidence  that  the  model 
of  proof  checking  has  some  psychological  validity,  in  the  sense  that  it 
simulates  a  cognitive  procedure  that  human  problem  solvers  can  learn  to 
perform. 
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Materials*  Instructional  materials  were  designed  for  presentation 
to  small  groups  of  student  subjects.  A  description  was  given  of  a 
five-step  procedure,  referred  to  as  the  "if-then”  procedure  because  of 
its  emphasis  on  the  antecedent-consequent  structure  of  reasons  for  proof 
steps.  The  steps  are:  (1)  Check  if  the  reason  is  a  valid  theorem, 
postulate,  or  definition;  (2)  Divide  the  reason  into  its  "if"  and 
"then"  components;  (3)  Check  the  antecedent:  has  it  previously  been 
shown  in  the  given  information,  the  diagram,  or  in  previous  steps  of  the 
proof?  (4)  Check  the  consequent:  does  it  match  the  relationship  of  the 
statement?  (5)  If  the  statement  is  the  last  one  in  the  proof,  does  it 
match  the  goal  of  the  problem?  Note  that  the  steps  of  the  procedure 
correspond  to  the  steps  shown  in  Table  8,  except  that  instantiation  of 
the  reason  is  left  implicit.  In  the  instruction,  instantiation  was 
shown  implicitly  in  examples,  in  which  the  antecedents  and  consequents 
of  reasons  were  converted  to  appropriate  propositions  about  the  objects 
in  the  problems. 

After  the  procedure  was  explained,  two  example  problems  were  worked 
by  the  group  of  student  subjects  working  together,  with  supervision  by 
the  instructor-experimenter.  Then  booklets  were  provided  that  contained 
proof-checking  problems.  On  each  problem,  the  students  first  worked 
Individ ually  on  the  problem,  then  one  student  was  asked  to  check  the 
proof  aloud,  going  through  the  "if-then"  procedure  for  each  line  of  the 
proof.  Any  errors  or  omissions  were  corrected  by  the  instructor,  and 
questions  asked  by  the  students  were  discussed.  Two  booklets  of 
proof -checking  exercises  were  prepared  for  use  in  two  training  sessions. 


In  addition  to  the  materials  for  instruction  in  proof  checking, 
there  also  were  materials  that  reviewed  basic  concepts  of  geometry  and 
construction  of  proofs.  Initially,  the  instructor  reviewed  basic 
definitions,  postulates,  and  theorems  needed  to  solve  proof  problems 
involving  congruence  of  triangles.  Booklets  were  provided  that 
contained  problems  of  constructing  proofs,  and  students  worked 
individually,  then  reviewed  each  problem,  as  described  previously.  Two 
booklets  of  problems  in  proof  construction  were  prepared  for  use  in  two 
review  sessions.  All  problems  used  in  the  review  of  proof  construction 
and  in  training  of  proof  checking  were  in  the  domain  of  congruent 
triangles . 

Materials  also  were  prepared  for  assessing  che  results  of 
Instruction.  A  set  of  problems  involving  congruence  of  triangles  was 
prepared  as  a  test,  designed  to  be  given  individually  to  the  students. 
The  test  problems  included  10  proof-checking  problems  and  three  problems 
of  constructing  proofs. 

Finally,  materials  were  prepared  for  testing  transfer.  These 
Included  a  brief  review  of  definitions,  postulates,  and  theorems 
involving  angles  formed  by  parallel  lines  intersected  by  a  transversal. 
These  concepts  and  propositions  had  not  been  included  in  the  previous 
instruction  or  testing,  and  there  was  no  discussion  of  proof  checking 
during  the  review  of  concepts  about  parallel  lines.  A  transfer  test  was 
presented,  consisting  of  11  proof-checking  problems  and  three  problems 
of  constructing  proofs,  all  concerned  with  angles  formed  by  parallel 
lines  intersected  by  a  transversal. 


Subjects  and  Procedure .  Subjects  were  30  students  at  the 
University  of  Pittsburgh  who  had  studied  geometry  in  high  school  but 
were  not  majoring  either  in  mathematics  or  computer  science.  Subjects 
were  paid  for  their  participation  in  the  experiment. 

The  study  was  conducted  in  1-hour  sessions  on  consecutive  days. 
Subjects  in  an  experimental  condition  received  the  review  of  triangle 
congruence  and  proof  construction  in  two  group  sessions;  then  training 
in  checking  proofs  about  congruent  triangles  was  given  in  two  further 
group  sessions,  and  finally  the  test  problems  and  the  transfer  to 
parallel  lines  were  given  in  two  individual  sessions.  Subjects  in  a 
control  condition  received  the  two  review  sessions  and  the  test  and 
transfer  sessions,  without  the  intervening  instruction  in  proof 
checking.  Fifteen  subjects  (nine  women  and  six  men)  participated  in 
each  of  the  conditions. 

The  subjects  in  the  experimental  condition  participated  on  6 
consecutive  days;  subjects  in  the  control  condition  participated  on  4 
consecutive  days.  For  each  day  on  which  they  participated,  subjects 
chose  a  convenient  time  from  among  a  number  of  available  sessions. 
Group  sessions  usually  had  three  or  four  students.  In  the  test  and 
transfer  sessions,  involving  individual  subjects,  students  were  asked  to 
think  aloud  as  they  worked  on  problems,  and  their  protocols  were 


recorded  on  audio  tape 


Results .  Performance  of  the  students  in  the  control  and 
experimental  conditions  is  summarized  in  Table  9.  The  instruction 
apparently  had  a  positive  effect,  enabling  students  in  the  experimental 
condition  to  detect  a  higher  proportion  of  errors  than  the  control 
students.  The  difference  in  performance  on  invalid  proofs  was 
significant  (£(28)  -  2.35,  £  <  .05),  and  there  clearly  was  not  an 
interaction  between  training  conditions  and  sessions.  The  data  in  Table 
9  are  based  on  strict  scoring  of  error  detection,  in  which  errors  had  to 
be  identified  and  the  reasons  for  the  errors  given  correctly.  Data  also 
were  tabulated  using  a  more  lenient  criterion,  where  errors  had  to  be 
identified  correctly  but  the  subject's  explanation  of  the  error  was 
incorrect  in  some  way.  With  the  lenient  criterion,  there  was  a 
significant  main  effect  of  training  (t(28)  =  2.89,  p  <  .01)  and  a 
nonsignificant  interaction  with  sessions  (t(28)  *  0.59,  p  >  .50),  as 
with  the  strict  criterion. 

An  examination  of  the  individual  problems  failed  to  reveal  any 
systematic  differences  in  the  kinds  of  problem  for  which  the  instruction 
in  proof  checking  was  especially  effective.  For  example,  some  errors  in 
proof  involve  incorrect  reasons;  others  involve  antecedents  that  have 
not  been  established.  There  was  a  somewhat  larger  effect  of  instruction 
on  problems  with  incorrect  reasons  in  the  test  session,  but  the  effect 
on  problems  with  antecedents  not  established  was  greater  in  the  transfer 
session.  One  specific  problem  for  which  instruction  had  an  especially 
large  effect  is  the  problem  shown  in  Figure  6,  which  was  included  in  the 
transfer  session.  One  of  the  15  Control  subjects  detected  the  error  in 
this  problem  correctly,  whereas  eight  of  the  experimental  subjects  did. 


Table  9 


Transfer 


The  thinking-aloud  protocols  were  examined  to  determine  whether  the 
subjects  used  the  proof-checking  procedure  in  an  explicit  way.  Three  of 
the  subjects  in  the  experimental  condition  used  the  procedure  on  nearly 
all  of  the  steps  in  every  problem.  Most  of  the  students  in  the 
experimental  conditions  applied  the  procedure  when  they  appeared  to  be 
uncertain  about  a  step.  Only  two  of  the  subjects  in  the  control 
condition  used  a  form  of  the  proof-checking  procedure  that  included 
explicit  checking  of  the  antecedents  and  consequents  of  reasons. 

The  experimenters  formed  two  general  impressions  of  differences 
between  subjects  in  the  two  conditions.  First,  experimental  subjects 
appeared  to  read  the  statements  of  the  proof  more  carefully  than  did  the 
control  subjects,  even  when  the  experimental  subjects  were  not  using  the 
proof -checking  procedure  explicitly.  Second,  the  control  participants 
typically  checked  steps  by  comparing  the  statements  with  the  problem 
diagrams.  Only  one  student  in  the  experimental  condition  appeared  to 
use  the  strategy  of  comparing  statements  with  diagrams.  The  rest 
attended  primarily  to  the  reason  and  related  it  to  information  from 
previous  steps  in  the  proof . 

There  was  a  substantial  difference  between  the  two  conditions  in 
performance  on  proof  construction  problems  in  both  the  test  and  transfer 
sessions.  The  difference  was  significant  (£(28)  =*  2.94,  £  <  .01)  and 
the  interaction  with  sessions  was  not  significant  (£(28)  ■  1.02,  £  > 
.50).  A  substantial  part  of  the  difference  was  due  to  one  problem  in 
the  test  session  that  used  a  pattern  of  overlapping  triangles  on  which 
experimental  subjects  had  some  specific  practice  that  was  not  given  to 
control  subjects.  Even  without  that  problem,  however,  there  was  an 


advantage  for  the  experimental  subjects:  .43  to  .30  on  proof 
construction  problems.  It  seems  likely  that  the  training  in  proof 
checking  gave  subjects  some  skills  that  facilitated  their  performance  in 
proof  construction  problems  as  well. 

Discussion.  The  subjects  who  were  given  training  in  proof  checking 
did  not  become  experts  on  that  task,  but  they  clearly  acquired  some 
skill  that  subjects  without  that  training  lacked.  The  result  provides 
general  support  for  the  computational  model  of  proof  checking,  showing 
that  it  is  learnable  at  least  to  an  extent.  The  fact  that  students 
still  failed  to  detect  some  errors  may  be  attributable  simply  to  the 
relatively  small  amount  of  training,  combined  with  the  absence  of  any 
constraints  on  the  subjects  to  use  the  procedure  they  had  learned  in  the 
test  and  transfer  problems.  As  a  methodological  point,  the  experiment 
has  the  interesting  feature  that  human  problem  solvers  were  trained  to 
perform  in  agreement  with  a  model  that  existed  earlier;  thus,  we 
succeeded  in  getting  human  performance  to  simulate  a  computer  program, 
rather  than  the  other  way. 

An  important  conceptual  question  is  whether  the  skill  represented 
in  the  computational  model  and  acquired  to  some  extent  by  the  student 
subjects  constitutes  significant  understanding  of  the  concept  of  proof. 
It  seems  to  us  that  it  does.  Knowledge  of  the  procedure  enables  an 
individual  to  analyze  relations  between  each  step  of  a  proof  and  the 
information  in  previous  statements  and  the  premises  of  the  problem.  The 
relation  that  is  examined  is  deductive  consequence,  the  defining 
characteristic  of  formal  proof.  It  seems  reasonable  to  characterize  a 
procedure  for  testing  the  features  of  deductive  consequence  as  a  form  of 
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significant  implicit  understanding  of  the  concept  of  formal  proof 


The  view  that  ability  to  check  proofs  reflects  understanding  of  the 
principle  of  deductive  consequence  is  strengthened  by  contrasting 
correct  performance  on  proof  checking  with  performance  by  students  who 
have  not  received  instruction  in  the  task.  Typical  untutored 
performance  strongly  resembles  comprehension  of  narrative  or  expository 
text,  where  each  new  statement  is  accepted  if  it  is  consistent  with 
previous  information  and  can  be  added  to  it  in  a  coherent  structure.  In 
a  proof,  a  stronger  criterion  of  acceptance  should  be  applied  to  each 
new  statement:  It  must  not  only  be  consistent  with  prior  information, 
it  must  follow  from  it  deductively.  The  procedure  for  checking  applies 
this  stronger  criterion,  and  knowledge  of  the  procedure  therefore 
provides  a  form  of  knowledge  that  relates  directly  to  characteristics 
that  distinguish  formal  proof  from  ordinary  texts. 


Conclusions 

In  the  introduction  to  this  chapter  I  mentioned  four  general  points 
that  are  illustrated  by  aspects  of  the  analyses  that  I  have  discussed. 
I  now  return  to  those  points  as  a  framework  for  presenting  some 
conclusions . 

First,  the  analyses  illustrate  the  applicability  of  methods  of 
analysis  developed  in  cognitive  science  to  school  tasks,  and  show  how 
those  methods  can  lead  to  formulation  of  cognitive  objectives  of 
instruction.  The  analyses  summarized  by  Resnick  (in  press)  involving 
reading,  physics,  and  elementary  mathematics,  and  the  investigations 


reported  by  Bereiter  and  Scardamalia  (in  press)  involving  writing, 
should  leave  little  doubt  as  to  the  feasibility  of  research  into  the 
cognitive  requirements  of  significant  instructional  tasks  using 
currently  available  concepts  and  methods  of  cognitive  science.  Of 
course,  this  should  not  be  interpreted  as  suggesting  that  more  powerful 
and  valid  concepts  and  methods  will  not  be  developed  in  the  future,  but 
only  that  significant  and  useful  insights  can  be  obtained  with  the  tools 
that  we  have  at  present. 

These  analyses  also  provide  characterizations  of  knowledge  that  can 
be  adopted  as  definite  objectives  of  instruction.  The  analysis  of 
structural  understanding  given  here  identifies  a  specific  cognitive 
structur  of  general  relationships  as  the  knowledge  that  constitutes 
understanding  of  the  structure  of  a  class  of  geometry  problems.  The 
analysis  of  understanding  of  the  principle  of  deductive  consequence 
identifies  a  cognitive  procedure  that  incorporates  defining  features  of 
valid  deductive  inference.  Both  these  characterizations  could  provide 
objectives  for  instruction  that  are  specific  enough  to  be  incorporated 
into  instructional  materials,  if  it  is  thought  that  their  acquisition 
would  be  valuable.  The  second  general  point  involves  analysis  of 
acquisition  and  instructional  intervention.  In  the  example  of 
structural  understanding,  a  theoretical  analysis  of  acquisition  was 
developed.  This  analysis  was  ad  hoc  in  many  ways,  and  much  work  remains 
for  the  development  of  a  general  model  of  schema-based  learning.  Even 
so,  some  suggestions  for  instruction  can  be  taken  from  the  analysis.  A 
major  condition  for  meaningful  learning  to  occur  in  the  model  is  the 
activation  of  an  appropriate  schema  in  the  learner's  knowledge 
structure.  This  suggests  that  at  a  minimum,  instruction  with  the  goal 
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of  structural  understanding  should  include  an  effort  to  activate  general 
schematic  knowledge  that  can  provide  an  appropriate  structure  for  the 
material  being  learned.  A  straightforward  method  would  Involve  simple 
discussion  of  the  structure  involved;  in  the  case  of  the  part-whole 
schema,  pointing  out  to  students  that  segments  or  angles  in  the  problems 
have  other  segments  or  angles  as  their  parts,  and  discussing  the  way  in 
which  these  part-whole  structures  are  involved  in  the  inferences  that 
are  made  in  solving  the  problems.  Another  Instructional  method  involves 
structural  mapping,  where  an  analogy  is  used  involving  the  problem  that 
is  the  target  of  instruction  and  a  problem  from  another  domain  that  has 
the  same  structure  in  a  salient  form.  The  efficacy  of  analogical 
mapping  for  geometry  problems  has  not  been  studied  systematically;* 
however,  its  general  usefulness  is  widely  recognized,  and  it  has  been 
studied  systematically  in  other  domains  such  as  electricity  and 
electronics  (Gentner,  1980;  Riley,  Bee,  &  Mokwa,  1981)  and  in 
elementary  mathematics  (Resnick,  in  press). 


In  the  example  of  understanding  the  principle  of  deductive 
consequence,  instructional  materials  were  developed  for  teaching  the 


*1  can  report  an  anecdote  in  which  the  method  of  analogical  mapping 
provided  a  successful  instructional  experience  for  the  vertical  angles 
problem.  An  eighth-grade  student  was  shown  the  proof  of  the  theorem, 
but  was  unable  to  recall  it  about  two  weeks  later.  Two  analogous 
situations  were  then  discussed.  One  of  these  involved  distances  on  a 
map,  with  two  equal  total  distances  and  a  shared  component,  and  the 
conclusion  of  equal  partial  distances  with  the  shared  component  removed. 
The  other  situation  involved  a  person  standing  on  a  scale  holding  one  of 
two  filled  suitcases;  the  combined  weights  of  the  person  with  either  of 
the  suitcases  was  equal,  and  the  conclusion  was  that  the  suitcases  had 
to  be  equal.  With  these  items  of  background,  the  student  generated  the 
proof  of  equal  vertical  angles  and  remembered  it  on  two  later  occasions, 
one  2  days  later  and  the  other  7  months  later. 


procedure  of  proof  checking  that  incorporates  defining  features  of  valid 
deductive  inference.  Instruction  using  the  materials  was  modestly 
successful,  and  because  we  gave  only  2  hours  of  instruction,  it  seems 
quite  likely  that  sustained  use  of  the  method  in  a  geometry  course  would 
have  substantial  benefit  for  students.  This  example,  involving  an 
instructional  objective  in  procedural  form,  is  particularly  adaptable 
for  instructional  purposes,  as  a  procedure  can  both  be  taught  and 
assessed  in  a  straightforward  way. 

The  third  general  point  involves  theoretical  significance  of  the 
analyses.  Development  of  definite  models  as  hypotheses  about  cognitive 
structures  that  constitute  understanding  should  provide  some 
clarification  of  understanding  in  relation  to  general  concepts  of 
psychological  theory. 

The  major  psychological  concept  involved  in  the  example  of 
structural  understanding  is  that  of  a  schema.  This  concept  has  been 
central  in  the  theory  of  language  understanding  that  has  been  developed 
recently  (e.g.,  Norman  &  Rumelhart,  1975),  and  it  would  be  reasonable  to 
expect  that  it  would  also  be  useful  in  the  analysis  of  understanding  in 
nonlinguistic  domains.  A  satisfying  conceptual  continuity  is  provided 
by  the  finding  that  a  schema  organized  in  essentially  the  same  way  as 
those  hypothesized  in  analyses  of  language  understanding  and  knowledge 
representation  (Bobrow  &  Winograd,  1977)  can  provide  a  plausible  account 
of  structural  understanding  of  a  class  of  mathematical  problems. 


The  theoretical  contribution  of  the  analysis  of  understanding  the 
principle  of  deductive  consequence  involves  a  characterization  of 
implicit  understanding.  We  attribute  implicit  understanding  of  a 
principle  to  an  individual  when  the  principle  plays  a  significant 
functional  role  in  the  individual's  performance.  One  way  in  which  a 
principle  can  be  functional  is  in  a  procedure  for  evaluating  examples  to 
determine  whether  they  satisfy  the  principle.  The  procedure  for  proof 
checking  that  was  formulated  and  that  students  acquired  in  instruction 
evaluates  solutions  of  proof  problems  according  to  the  principle  of 
deductive  consequence,  and  thus  illustrates  this  form  of  implicit 
understanding. 

Using  performance  in  an  evaluation  task  such  as  proof  checking  as  a 
criterion  for  knowledge  of  a  general  concept  has  two  important 
precedents.  First,  in  experimental  studies  of  concept  identification, 
subjects'  ability  to  distinguish  correctly  between  positive  and  negative 
examples  is  taken  as  the  criterion  of  their  acquisition  of  the  concept. 
In  a  more  complex  domain,  knowledge  of  the  grammar  of  a  language  is 
tested  by  the  ability  of  a  human  subject  or  computational  system  to 
discriminate  correctly  between  strings  that  are  sentences  of  the 
language  and  strings  that  are  not  sentences  according  to  the  grammar. 

The  final  general  point  mentioned  in  the  introduction  was  that 
"understanding"  refers  to  numerous  distinct  forms  of  knowledge.  The 
examples  discussed  in  this  chapter  illustrate  two  major  categories  of 
understanding:  intrinsic  and  theoretical  understanding.  Intrinsic 
understanding  of  a  problem  involves  cognizance  of  relationships  among 
elements  of  the  problem  or  steps  in  its  solution.  Theoretical 


understanding  involves  cognizance  of  relationships  between  the  problem 
and  general  principles  that  constrain  or  justify  aspects  of  the 
solution.  (A  more  extended  discussion  of  these  forms  of  understanding 
has  been  given  by  Greeno  &  Riley,  1981.)  Structural  understanding,  in 
the  characterization  given  here,  is  a  form  of  intrinsic  understanding, 
involving  cognizance  of  a  set  of  relationships  among  problem  components. 
Understanding  the  principle  of  deductive  consequence  is  an  example  of 
theoretical  understanding,  involving  cognizance  of  a  significant 
constraint  on  valid  solutions  of  proof  problems. 

The  difference  between  these  two  forms  of  understanding  emphasizes 
the  importance  of  identifying  the  cognitive  structures  and  processes 
that  are  intended  when  we  ask  whether  someone  understands  a  problem  or  a 
procedure.  The  cognitive  structures  identified  in  each  of  the  analyses 
constitute  significant  understanding  of  proof  problems.  We  would  say 
that  a  student  with  cognizance  of  the  part-whole  relations  in  the 
vertical  angles  problem  has  more  understanding  of  that  problem  than  a 
student  without  that  cognizance.  Similarly,  a  student  with  cognizance 
of  the  defining  features  of  valid  deductive  inference  has  more 
understanding  of  any  proof  problem  than  a  student  without  that 
cognizance.  Furthermore,  there  are  other  significant  forms  of 
understanding  in  the  domain  of  proof  problems  that  could  be  taken  into 
account  in  a  theoretical  analysis  or  in  design  of  instruction. 

Although  an  undifferentiated  concept  of  understanding  is  inadequate 
for  both  theoretical  and  instructional  purposes,  a  more  differentiated 
and  specific  characterization  of  understanding  is  both  important  and 
feasible.  Concepts  and  methods  of  cognitive  psychology  have  now  been 
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developed  chat  make  it  possible  to  formulate  objectives  for  instruction 
that  are  specific  enough  to  be  used  as  the  basis  of  instructional  design 
and  assessment,  and  that  also  correspond  to  significant  forms  of 
understanding. 


References 


Anderson,  J.  R,  Language,  memory,  and  thought.  Hillsdale,  N.J.: 
Lawrence  Erlbaum  Associates,  1976. 

Anderson,  J.  R.  Acquisition  of  cognitive  skill.  (Technical  Report 
81-1).  Pittsburgh:  Carnegie-Mellon  University  Department  of 
Psychology.  1981. 

Anderson,  J.  R. ,  Greeno,  J.  G.,  Kline,  P.  J.,  &  Neves,  D.  M. 

Acquisition  of  problem-solving  skill.  In  J.  R.  Anderson  (Ed.), 
Cognitive  skills  and  their  acquisition.  Hillsdale,  N.J.:  Lawrence 
Erlbaum  Associates,  1981. 

Bereiter,  C.,  &  Scardamalia,  M.  From  conversation  to  composition:  The 
role  of  instruction  in  a  developmental  process.  In  R.  Glaser 
(Ed.),  Advances  in  instructional  psychology  (Vol.2).  Hillsdale, 
N.J.:  Lawrence  Erlbaum  Associates,  1982. 

Bobrow,  D.  G.,  &  Winograd,  T.  An  overview  of  KRL,  a  knowledge 
representation  language.  Cognitive  Science,  1977,  1_,  3-46. 

Gentner,  D.  The  structure  of  analogocal  models  in  science.  (Report  No. 
4451).  Cambridge,  Mass.:  Bolt,  Beranek,  &  Newman,  1980. 

Gelman,  R.,  &  Galllstel,  C.  R.  The  child's  understanding  of  number. 
Cambridge,  Mass.:  Harvard  University  Press,  1978. 


43 


Greeno,  J.  G.  1976.  Indefinite  goals  in  well-structured  problems. 
Psychological  Review,  1976,  J32,  479-491. 

Greeno,  J.  G.  A  study  of  problem  solving.  In  R.  Glaser  (Ed.),  Advances 
in  Instructional  psychology  (Vol.  1).  Hillsdale,  N.  J.:  Lawrence 
Erlbaum  Associates,  1978. 

Greeno,  J.  G.,  Magone,  M.  E.,  &  Chaiklin,  S.  Theory  of  constructions 
and  set  in  problem  solving.  Memory  &  Cognition,  1979,  7_,  445-461. 

Greeno,  J.  G.,  &  Riley,  M.  S.  Processes  and  development  of 

understanding.  (Report  1981/8).  Pittsburgh:  University  of 
Pittsburgh  Learning  Research  and  Development  Center,  1981. 

Judd,  C.  M.  The  relation  of  special  training  to  general  intelligence. 
Educational  Review,  1908‘,  36,  2842. 

Katona,  G.  Organizing  and  memorizing.  New  York:  Columbia  University 
Press,  1940. 

Kintsch,  W.,  &  VanDijk,  T.  A.  Toward  a  model  of  text  comprehension  and 
production.  Psychological  Review,  1978,  85,  363-394. 

Neves,  D.  M.  Learning  procedures  from  examples.  Unpublished  doctoral 
dissertation,  Carnegie-Mellon  University,  1981. 

Norman,  D.  A.,  &  Rumelhart,  D.  E.  Explorations  in  cognition.  San 
Francisco:  W.  H.  Freeman,  1975. 


44 


Piaget,  J.  The  child's  conception  of  number.  New  York:  W.  W. 

Norton,  1965.  (Originally  published  1941). 

Resnick,  L.  B.  A  developmental  theory  of  number  understanding.  In  H. 
P.  Gins burg  (Ed.),  Development  of  mathematical  thinking.  New 
York:  Academic  Press,  in  press. 

Riley,  M.  S.,  Bee,  N.  V.,  &  Mokwa,  J.  J.  Representations  in  early 
learning:  The  acquisition  of  problem-solving  strategies  in  basic 
electricity/electronics .  Pittsburgh:  University  of  Pittsburgh 
Learning  Research  and  Development  Center,  1981. 

Riley,  M.  S.,  Greeno,  J.  G.,  &  Heller,  J.  I.  Development  of  word 

problem  solving  ability.  In  H.  P.  Ginsburg  (Ed.),  Development  of 
mathematical  thinking.  New  York:  Academic  Press,  in  press. 

Sacerdoti,  E.  D.  A  structure  for  plans  and  behavior.  New  York: 
Elsevier-North  Holland  Publishing  Co.,  1977. 

Vere,  S.  A.  Inductive  learning  of  relational  productions.  In  D.  A. 
Waterman  &  F.  Hayes-Roth  (Eds.),  Pattern-directed  Inference 
systems .  New  York:  Academic  Press,  1978. 

Wertheimer,  M.  Productive  thinking  (Enlarged  ed.).  New  York:  Harper  & 


Row,  1959.  (Originally  published  1945.) 


Acknowledgements 


This  research  was  supported  by  the  Personnel  and  Training  Research 
Programs,  Office  of  the  Naval  Research,  under  Contract  Number 
N00014-79-C-0215.  Contract  Authority  Identification  Number  NR  667-430. 
The  study  of  proof  checking  was  a  collaborative  study  by  Maria  Magone 
and  me.  James  Rowland  assisted  in  collecting  and  analyzing  the  data  of 
the  instructional  experiment,  and  James  Mokwa  assisted  in  implementing 
models . 

This  report  is  an  expanded  version  of  a  paper  presented  at  the  XXII 
International  Congress  of  Psychology  in  Leipzig,  in  July  1980.  The 
original  version  has  been  published  in  R.  Glaser  &  J.  Lompscher 
(Eds.),  Cognitive  and  motivational  aspects  of  instruction,  a  volume  of 
proceedings  of  the  Congress,  and  in  Japanese,  in  Psychology,  1982, 

60-64. 


PITTSBURGH/NECHES  August  3,  1982 


Page  1 


Navy 


1  Meryl  S.  Baker 
NPRDC 
Code  P309 

San  Diego,  CA  92152 

1  Dr.  Robert  Breaux 
Code  N-711 
NAVTRAEQUIPCEN 
Orlando,  FL  32813 

1  CDR  Mike  Curran 

Office  of  Naval  Research 
800  N.  Quincy  St. 

Code  270 

Arlington,  VA  22217 

1  DR.  PAT  FEDERICO 

NAVY  PERSONNEL  R&D  CENTER 
SAN  DIEGO,  CA  92152 

1  Dr.  John  Ford 

Navy  Personnel  R&D  Center 
San  Diego,  CA  92152 

1  LT  Steven  D.  Harris,  MSC,  USN 
Code  6021 

Naval  Air  Development  Center 
Warminster,  Pennsylvania  1897*1 

1  Dr.  Jim  Hollan 
Code  304 

Navy  Personnel  R&D  Center 
San  Diego,  CA  92152 

1  Dr.  Norman  J.  Kerr 

Chief  of  Naval  Technical  Training 
Naval  Air  Station  Memphis  (75) 
Millington,  TN  38054 

1  Dr.  William  L.  Maloy 

Principal  Civilian  Advisor  for 
Education  and  Training 
Naval  Training  Command,  Code  OOA 
Pensacola,  FL  32508 


Navy 


1  CAPT  Richard  L.  Martin,  USN 
Prospective  Commanding  Officer 
USS  Carl  Vinson  (CVN-70) 

Newport  News  Shipbuilding  and  Drydock  Co 
Newport  News,  VA  23607 

1  Dr.  George  Moeller 

Head,  Human  Factors  Dept. 

Naval  Submarine  Medical.  Research  Lab 
Groton,  CN  06340 

1  Dr  William  Montague 

Navy  Personnel  R&D  Center 
San  Diego,  CA  92152 

1  Ted  M.  I.  Yellen 

Technical  Information  Office,  Code  201 
NAVY  PERSONNEL  R&D  CENTER 
SAN  DIEGO,  CA  92152 

1  Library,  Code  P201L 

Navy  Personnel  R&D  Center 
San  Diego,  CA  92152 

1  Technical  Director 

Navy  Personnel  R&D  Center 
San  Diego,  CA  92152 

6  Commanding  Officer 

Naval  Research  Laboratory 
Code  2627 

Washington,  DC  20390 

1  Psychologist 

0NR  Branch  Office 
Bldg  114,  Section  D 
666  Summer  Street 
Boston,  MA  02210 

1  Office  of  Naval  Research 
Code  437 

800  N.  Quincy  SStreet 
Arlington,  VA  22217 

5  Personnel  &  Training  Research  Programs 
(Code  458) 

Office  of  Naval  Research 
Arlington,  VA  22217 


PITTSBURGH/NECHES  August  3,  1982 


Page  2 


Navy 


1  Psychologist 

ONR  Branch  Office 
1030  East  Green  Street 
Pasadena,  CA  91101 

1  Special  Asst,  for  Education  and 
Training  (0P-01E) 

Rm.  2705  Arlington  Annex 
Washington,  DC  20370 


1  LT  Frank  C.  Petho,  MSC,  USN  (Ph.D) 

Selection  and  Training  Research  Division 
Human  Performance  Sciences  Dept. 

Naval  Aerospace  Medical  Research  Laborat 
Pensacola,  FL  32508 

1  Dr.  Gary  Poock 

Operations  Research  Department 
Code  55PK 

Naval  Postgraduate  School 
Monterey,  CA  93940 

1  Roger  W.  Remington,  Ph.D 
Code  L52 
NAMRL 

Pensacola,  FL  32508 

1  Dr.  Worth  Scanland,  Director 

Research,  Development,  Test  &  Evaluation 
N-5 

Naval  Education  and  Training  Command 
NAS,  Pensacola,  FL  32508 

1  Dr.  Alfred  F.  Smode 

Training  Analysis  &  Evaluation  Group 
(TAEG) 

Dept,  of  the  Navy 
Orlando,  FL  32813 

1  Dr.  Richard  Sorensen 

Navy  Personnel  RAD  Center 
Sen  Diego,  CA  92152 


Navy 


1  Roger  Weissinger-Baylon 

Department  of  Administrative  Sciences 
Naval  Postgraduate  School 
Monterey,  CA  93940 

1  Dr.  Robert  Wisher 
Code  309 

Navy  Personnel  RAD  Center 
San  Diego,  CA  92152 

Mr  John.  H.  Wolfe 
Code  P310 

U.  S.  Navy  Personnel  Research  and 
Development  Center 
San  Diego,  CA  92152 


1  Office  of  the  Chief  of  Naval  Operations  1 
Research  Development  A  Studies  Branch 
(OP-115) 

Washington,  DC  20350 
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Army 


Air  Force 


1  Technical  Director 

U.  S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

1  Mr.  James  Baker 

Systems  Manning  Technical  Area 
Army  Research  Institute 
5001  Eisenhower  Ave. 

Alexandria,  VA  22333 

1  Dr.  Beatrice  J.  Farr 

U.  S.  Army  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

1  Dr.  Michael  Kaplan 

U.S.  ARMY  RESEARCH  INSTITUTE 
5001  EISENHOWER  AVENUE 
ALEXANDRIA,  VA  22333 

1  Dr.  Milton  S.  Katz 

Training  Technical  Area 
U.S.  Army  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

1  Dr.  Harold  F.  O'Neil,  Jr. 

Attn:  PERI-OK 
Army  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

1  Dr.  Robert  Sasmor 

U.  S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

1  Dr.  Joseph  Ward 

U.S.  Army  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 


1  Dr.  Earl  A.  Alluisi 
HQ,  AFHRL  (AFSC) 

Brooks  AFB,  TX  78235 

1  Dr.  Genevieve  Haddad 
Program  Manager 

Life  Sciences  Directorate 
AF0SR 

Bolling  AFB,  DC  20332 

2  3700  TCHTW/TTGH  Stop  32 
Sheppard  AFB,  TX  76311 
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Other  DoD 


Civil  Govt 


12  Defense  Technical  Information  Center  1  Dr.  Paul  G.  Chapin 

Cameron  Station,  Bldg  5  Linguistics  Program 

Alexandria,  VA  22314  National  Science  Foundation 

Attn:  TC  Washington,  DC  20550 

1  Military  Assistant  for  Training  and  1  Dr.  Susan  Chipman 

Personnel  Technology  Learning  and  Development 

Office  of  the  Under  Secretary  of  Defense  National  Institute  of  Education 
for  Research  &  Engineering  1200  19th  Street  NW 

Room  3D129,  The  Pentagon  Washington,  DC  20208 

Washington,  DC  20301 

1  Dr.  Arthur  Melm-d 

1  DARPA  National  Intitute  of  Education 

1400  Wilson  Blvd.  1200  19th  Street  NW 

Arlington,  VA  22209  Washington,  DC  20208 

1  Dr.  Andrew  R.  Molnar 

Science  Education  Dev; 
and  Research 

National  Science  Foundation 
Washington,  DC  20550 

1  Dr.  Joseph  Psotka 

National  Institute  of  Education 
1200  19th  St.  NW 
Washington, DC  20208 

1  Dr.  Frank  Withrow 

U.  S.  Office  of  Education 
400  Maryland  Ave.  SW 
Washington,  DC  20202 

1  Dr.  Joseph  L.  Young,  Director 
Memory  A  Cognitive  Processes 
National  Science  Foundation 
Washington,  DC  20550 
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Mon  Govt 


Dr.  John  R.  Anderson- 
Department  of  Psychology 
Carnegie  Mellon  University 
Pittsburgh,  PA  15213 

Anderson,  Thomas  H. ,  Ph.D. 
Center  for  the  Study  of  Reading 
174  Children's  Research  Center 
51  Ger ty  Drive 
Champiagn,  IL  61320 

Dr.  John  Annett 
Department  of  Psychology 
University  of  Warwick 
Coventry  CV4  7AL 
ENGLAND 

DR.  MICHAEL  ATWOOD 
SCIENCE  APPLICATIONS  INSTITUTE 
40  DENVER  TECH.  CENTER  WEST 
7935  E.  PRENTICE  AVENUE 
ENGLEWOOD,  CO  80110 

1  psychological  research  unit 
Dept,  of  Defense  (Army  Office) 
Campbell  Park  Offices 
Canberra  ACT  2600,  Australia 

Dr.  Alan  Baddeley 
Medical  Research  Council 

Applied  Psychology  Unit 
15  Chaucer  Road 
Cambridge  CB2  2EF 
ENGLAND 

Dr.  Patricia  Baggett 
Department  of  Psychology 
University  of  Colorado 
Boulder,  CO  80309 

Ms.  Carole  A.  Bagley 
Minnesota  Educational  Computing 
Consortium 

2354  Hidden  Valley  Lane 
Stillwater,  MN  55082 


Non  Govt 


Dr.  Jonathan  Baron 
Dept.  of  Psychology 
University  of  Pennsylvania 
3813-15  Walnut  St.  T-3 
Philadlphia ,  PA  19104 

Mr  Avron  Barr 

Department  of  Computer  Science 
Stanford  University 
Stanford,  CA  94305 

Dr.  John  Bergan 
School  of  Education 
University  of  Arizona 
Tuscon  AZ  85721 

Liaison  Scientists 
Office  of  Naval  Research, 

Branch  Office  ,  London 
Box  39  FPO  New  York  09510 

Dr.  Lyle  Bourne 
Department  of  Psychology 
University  of  Colorado 
Boulder,  CO  80309 

Dr.  Robert  Brennan 

American  College  Testing  Programs 

P.  0.  Box  168 

Iowa  City,  IA  52240 

Dr.  John  S.  Brown 

XEROX  Palo  Alto  Research  Center 

3333  Coyote  Road 

Palo  Alto,  CA  94304 

Dr.  Bruce  Buchanan 
Department  of  Computer  Science 
Stanford  University 
Stanford,  CA  94305 

DR.  C.  VICTOR  BUNDERS0N 
WICAT  INC. 

UNIVERSITY  PLAZA,  SUITE  10 
1160  SO.  STATE  ST. 

OREM,  UT  84057 
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1  Dr.  Pat  Carpenter 

Department  of  Psychology 
Carnegie-Mellon  University 
Pittsburgh,  PA  15213 

1  Dr.  John  B.  Carroll 
Psychometric  Lab 
Univ.  of  No.  Carolina 
Davie  Hall  013A 
Chapel  Hill,  NC  27514 

1  Dr.  William  Chase 

Department  of  Psychology 
Carnegie  Mellon  University 
Pittsburgh,  PA  15213 

1  Dr.  Michel ine  Chi 

Learning  R  &  D  Center 
University  of  Pittsburgh 
3939  O'Hara  Street 
Pittsburgh,  PA  15213 

1  Dr.  William  Clancey 

Department  of  Computer  Science 
Stanford  University 
Stanford,  CA  94305 

1  Dr.  Allan  M.  Collins 

Bolt  Beranek  &  Newman,  Inc. 

50  Moulton  Street 
Cambridge,  Ma  02138 

1  Dr.  Lynn  A.  Cooper 
LRDC 

University  of  Pittsburgh 
3939  O'Hara  Street 
Pittsburgh,  PA  15213 

1  Dr.  Meredith  P.  Crawford 

American  Psychological  Association 
1200  17th  Street,  N.W. 

Washington,  DC  20036 

1  Dr.  Kenneth  B.  Cross 
Anacapa  Sciences,  Inc. 

P.0.  Drawer  Q 

Santa  Barbara,  CA  93102 


Non  Govt 


1  LC0L  J.  C.  Eggenberger 

DIRECTORATE  OF  PERSONNEL  APPLIED  RESEARC 
NATIONAL  DEFENCE  HQ 
101  COLONEL  BY  DRIVE 
OTTAWA,  CANADA  K1A  0K2 

1  Dr.  Ed  Feigenbaum 

Department  of  Computer  Science 
Stanford  University 
Stanford,  CA  94305 

1  Dr.  Richard  L.  Ferguson 

The  American  College  Testing  Program 

P.0.  Box  168 

Iowa  City,  IA  52240 

1  Mr.  Wallace  Feurzeig 

Bolt  Beranek  &  Newman,  Inc. 

50  Moulton  St. 

Cambridge,  MA  02138. 

1  Dr.  Victor  Fields 
Dept,  of  Psychology 
Montgomery  College 
Rockville,  MD  20850 

1  Dr.  John  R.  Frederiksen 
Bolt  Beranek  &  Newman 
50  Moulton  Street 
Cambridge,  MA  02138 

1  Dr.  Alinde  Friedman 

Department  of  Psychology 
University  of  Alberta 
Edmonton,  Alberta 
CANADA  T6G  2E9 

1  DR.  ROBERT  GLASER 
LRDC 

UNIVERSITY  OF  PITTSBURGH 
3939  O'HARA  STREET 
PITTSBURGH,  PA  15213 

1  Dr.  Marvin  D.  Glock 
217  Stone  Hall 
Cornell  University 
Ithaca,  NY  14853 
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Non  Govt 


OR.  JAMES  G.  GREENO 
LRDC 

UNIVERSITY  OF  PITTSBURGH 
3939  O'HARA  STREET 
PITTSBURGH,  PA  15213 

Or.  Barbara  Hayes-Roth 
The  Rand  Corporation 
1700  Main  Street 
Santa  Monica,  CA  90406 

Dr.  Frederick  Hayes-Roth 
The  Rand  Corporation 
1700  Main  Street 
Santa  Monica,  CA  90406 

Dr.  Kristina  Hooper 
Clark  Kerr  Hall 
University  of  California 
Santa  Cruz,  CA  95060 

Glenda  Greenwald,  Ed. 
iHuman  Intelligence  Newsletter" 
P.  0.  Box  1163 
Birmingham,  MI  48012 

Dr.  Earl  Hunt 
Dept,  of  Psychology 
University  of  Washington 
Seattle,  WA  98105 


Non  Govt 


1  Dr.  Stephen  Kosslyn 
Harvard  University 
Department  of  Psychology 
33  Kirkland  Street 
Cambridge,  MA  02138 

1  Dr.  Marcy  Lansman 

Department  of  Psychology,  NI  25 
University  of  Washington 
Seattle,  WA  98195 

1  Dr.  Jill  Larkin 

Department  of  Psychology 
Carnegie  Mellon  University 
Pittsburgh,  PA  15213 

1  Dr.  Alan  Lesgold 
Learning  R&D  Center 
University  of  Pittsburgh 
Pittsburgh,  PA  15260 

1  Dr.  Erik  McWilliams 

Science  Education  Dev.  and  Research 
National  Science  Foundation 
Washington,  DC  20550 

1  Dr.  Mark  Miller 

TI  Computer  Science  Lab 
C/0  2824  Winterplace  Circle 
Plano,  TX  75075 


Dr.  Ed  Hutchins 

Navy  Personnel  R&D  Center 

San  Diego,  CA  92152 


Dr.  Greg  Kearsley 
HumRRO 

300  N.  Washington  Street 
Alexandria,  VA  22314 

Dr.  Walter  Kintsch 
Department  of  Psychology 
University  of  Colorado 
Boulder,  CO  80302 

Dr.  David  Kieras 
Department  of  Psychology 
University  of  Arizona 
Tuscon,  A2  85721 


1  Dr.  Allen  Munro 

Behavioral  Technology  Laboratories 
1845  Elena  Ave.,  Fourth  Floor 
Redondo  Beach,  CA  90277 

1  Dr.  Donald  A  Norman 

Dept,  of  Psychology  C-009 
Univ.  of  California,  San  Diego 
La  Jolla,  CA  92093 

1  Committee  on  Human  Factors 
JH  811 

2101  Constitution  Ave.  NW 
Washington,  X  20418 
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Non  Govt 


1  Dr.  Seymour  A.  Papert 

Massachusetts  Institute  of  Technology 
Artificial  Intelligence  Lab 
545  Technology  Square 
Cambridge,  MA  02139 

1  Dr.  James  W.  Pellegrino 
University  of  California, 

Santa  Barbara 
Dept,  of  Psychology 
Santa  Barabara,  CA  93106 

1  MR.  LUIGI  PETRULLO 

2431  N.  EDGEW00D  STREET 
ARLINGTON,  VA  22207 

1  DR.  PETER  POLSON 
DEPT.  OF  PSYCHOLOGY 
UNIVERSITY  OF  COLORADO 
BOULDER,  CO  80309 

1  MINRAT  M.  L.  RAUCH 
P  II  4 

BUNDESMINISTERIUM  DER  VERTEIDIGUNG 

POSTFACH  1328 

D-53  BONN  1,  GERMANY 

1  Dr.  Fred  Reif 
SESAME 

c/o  Physics  Department 
University  of  California 
Berkely,  CA  94720 

1  Mary  Riley 
LRDC 

University  of  Pittsburgh 
3939  O’Hara  Street 
Pittsburgh,'  PA  15213 

1  Dr.  Andrew  M.  Rose 

American  Institutes  for  Research 
1055  Thomas  Jefferson  St.  NW 
Washington,  DC  20007 

1  Dr.  Ernst  Z.  Rothko pf 
Bell  Laboratories 
600  Mountain  Avenue 
Murray  Hill,  NJ  07974 


Non  Govt 


1  Dr.  David  Rumelhart 

Center  for  Human  Information  Processing 
Univ.  of  California,  San  Diego 
La  Jolla,  CA  92093 

1  Dr.  Alan  Schoenfeld 

Department  of  Mathematics 
Hamilton  College 
Clinton,  NY  13323 

1  DR.  ROBERT  J.  SEIDEL 

INSTRUCTIONAL  TECHNOLOGY  GROUP 
HUMRRO 

300  N.  WASHINGTON  ST. 

ALEXANDRIA,  VA  22314 

1  Committee  on  Cognitive  Research 
%  Dr.  Lonnie  R.  Sherrod 
Social  Science  Research  Council 
605  Third  Avenue 
New  York,  NY  10016 

1  Robert  S.  Siegler 
Associate  Professor 
Carnegie-Mellon  University 
Department  of  Psychology 
Schenley  Park 
Pittsburgh,  PA  15213 

1  Dr.  Edward  E.  Smith 

Bolt  Beranek  A  Neuman,  Inc. 

50  Moulton  Street 
Cambridge,  MA  02138 

1  Dr.  Robert  Smith 

Department  of  Computer  Science 
Rutgers  University 
New  Brunswick,  NJ  08903 

1  Dr.  Richard  Snow 
School  of  Education 
Stanford  University 
Stanford ,  CA  94305 

1  Dr.  Robert  Sternberg 
Dept,  of  Psychology 
Yale  University 
Box  11A,  Yale  Station 
New  Haven,  CT  06520 


1  DR.  ALBERT  STEVENS 

BOLT  BERANEK  A  NEWMAN,  INC. 
50  MOULTON  STREET 
CAMBRIDGE,  MA  02138 


David  E.  Stone,  Ph.D. 

Hazeltine  Corporation 
7680  Old  Springhouse  Road 
McLean,  VA  22102 

DR.  PATRICK  SUPPES 

INSTITUTE  FOR  MATHEMATICAL  STUDIES  IN 
THE  SOCIAL  SCIENCES 
STANFORD  UNIVERSITY 
STANFORD,  CA  94305 


Dr.  Keith  T.  Weacourt 
Information  Sciences  Dept. 
The  Rand  Corporation 
1700  Main  St. 

Santa  Monica,  CA  90406 


1  Dr.  Kikuoi  Tatsuoka 

Computer  Based  Education  Research 
Laboratory 

252  Engineering  Research  Laboratory 
University  of  Illinois 
Urbana,  IL  61801 

1  Dr.  John  Thomas 

IBM  Thomas  J.  Watson  Research  Center 
P.0.  Box  218 

Yorlrtown  Heights,  NY  10598 


1  DR.  PERRY  THORNDYKE 
THE  RAND  CORPORATION 
1700  MAIN  STREET 
SANTA  MONICA,  CA  90406 

1  Dr.  Douglas  Towne 

Univ.  of  So.  California 
Behavioral  Technology  Labs 
1845  S.  Elena  Ave. 

Redondo  Beach,  CA  90277 


1  Dr.  Benton  J.  Underwood 
Dept,  of  Psychology 
'  •  Northwestern  University 
Evanston,  IL  60201 


1  DR.  GERSHON  WELTMAN 
PERCEPTRONICS  INC. 

6271  VARIEL  AVE. 

•  WOODLAND  HILLS,  CA  91367 


